191003 Meeting with STFC

# <img src="https://codimd.web.cern.ch/uploads/upload_45a14e417e9a8ade007f06e7b9420356.png" style="border: none;background: none;box-shadow:none"> <span style="color: MediumSeaGreen">*deployment status at CERN*</span> 03 October 2019 [Julien Leduc](mailto:julien.leduc@cern.ch) for the CTA team --- <h2>EOS+CTA <i style="color:blue;">Architecture</i></h2> <img src="https://codimd.web.cern.ch/uploads/upload_e764d94a4ee3ac79c328ea0d21a6a128.svg" class="plain"></span> --- ## Current Preproduction systems ---- ## Operating System *All servers* are currently running: ```bash [root@eosctafst0011 ~]# cat /etc/redhat-release CentOS Linux release 7.6.1810 (Core) [root@eosctafst0011 ~]# uname -r 3.10.0-957.5.1.el7.x86_64 ``` #### Provisionning <img src="https://codimd.web.cern.ch/uploads/upload_d3df55ec134a9d9e9885ffc2775b8ca6.png" class="plain" height=150px>+<img src="https://codimd.web.cern.ch/uploads/upload_8ca5945735d38161bcefc10b69461691.png" class="plain" height=150px> ----  ---- ## <span style="color: dodgerblue">EOSCTAPPS</span> Buffer <img src="https://codimd.web.cern.ch/uploads/upload_3f0cbd71fb908391309e4aa00f40cf74.svg" class="plain" height=60%> ---- ## <span style="color: dodgerblue">EOSCTA</span>PPS <img src="https://codimd.web.cern.ch/uploads/upload_88f3b03ec6cf37aa8c59787b8909d6f6.svg" class="plain"> ---- ## <span style="color: dodgerblue">EOSCTAPPS</span> Buffer <img src="https://codimd.web.cern.ch/uploads/upload_16f062d13b11e8413828870615825460.svg" class="plain" height=60%> --- ## ATLAS 2018 recall campaign CTA migration instance `eosctapps`: - 5 hyperconverged servers - 20TB of SSDs - 10 tape drives` ---- ## CTA cumulated recall volume <img src="https://codimd.web.cern.ch/uploads/upload_56d9d3f7ba73f6b59dc8b4616e1a2c4f.png" class="plain"> ---- ## CTA share in daily total recall volume <img src="https://codimd.web.cern.ch/uploads/upload_a3e2e74e123fd54914513bba10fe5c22.png" class="plain"> ---- ## CTA avg recall speed/hour <img src="https://codimd.web.cern.ch/uploads/upload_c24024734407c1aada74d3b1be28ae93.png" class="plain"> --- <h2>EOS+CTA <i style="color:dodgerblue;">Production instances</i></h2> Getting ready for the final migration: - final buffer hardware on its way: - racks are cabled - network switches allocated (600Gb/s of BW) - 30 hyper converged servers mid October - Production ready beginning of November ---- ```graphviz graph hierarchy { nodesep=1 // increases the separation between nodes node [color=Red, fontname=Courier, shape=box] //All nodes will this shape and colour edge [color=Blue, label="25Gb/s"] //All the lines look like this Router [shape=circle] Router--{SwitchBuffer} [label="3x(2x100Gb/s)", fontsize=15, style=bold] Router--{SwitchTape} [label="7x20Gb/s", fontsize=15, style=bold] subgraph cluster_level1{ label="EOSCTA Buffer infrastructure\n3x10 hyperconverged servers" color=dodgerblue fontcolor=dodgerblue SwitchBuffer SSD01 [color=black, shape=cylinder] SSDXX [color=black, shape=cylinder] SSD16 [color=black, shape=cylinder] buffersrv01 buffersrvXX--{SSD01 SSDXX SSD16} [label=""] } subgraph cluster_level2{ label="Tape infrastructure\nXX tapeservers" color=crimson fontcolor=crimson SwitchTape SwitchTape--{tpsrv01 tpsrvXX} [color=Blue, label="10Gb/s"] SwitchBuffer--{buffersrv01 buffersrvXX } [color=Blue, label="25Gb/s", style=bold] {rank=same; tpsrv01 tpsrvXX} // Put them on the same level tape [color=black, shape=Msquare] tpsrvXX--tape [label="360MB/s"] } } ``` --- ## eosctainstance demo instance A fully configured single node standalone eoscta instance. ---- ## Prerequisites - 1 Oracle database account - 1 Ceph objectstore pool - 1 `mhvtl` library with a few - kubernetes standalone installation ---- ## Getting Started In [cta repository](https://gitlab.cern.ch/cta/CTA): - [`cta/continuousintegration`](https://gitlab.cern.ch/cta/CTA/tree/master/continuousintegration) - launch it: ```bash cd continuousintegration/orchestration/ ./run_systemtest.sh -n testnamespace \ -s tests/archive_retrieve.sh \ -O -D -k ``` ---- {%vimeo 322128118 %}