# <img src="https://codimd.web.cern.ch/uploads/upload_45a14e417e9a8ade007f06e7b9420356.png" style="border: none;background: none;box-shadow:none"> initial deployments [Julien Leduc](mailto:julien.leduc@cern.ch) --- ## Data archiving at CERN <ul> <li class="fragment">Ad aeternum storage</li> <li class="fragment">7 tape libraries, 83 tape drives, 20k tapes</li> <li class="fragment">Current use: <b style="color:dodgerblue;">330 PB</b></li> <li class="fragment">Current capacity: <b style="color:coral;">0.7 EB</b></li> <li class="fragment"><b style="color:red;">Exponentially growing</b></li> </ul> <!-- .slide: data-background="https://codimd.web.cern.ch/uploads/upload_95716d3602c009e301c880b0afd4225a.png" data-background-size="80%" --> --- <h2>Data Archiving at CERN <span class="fragment"><i style="color:blue;">Evolution</i></span></h2> <ul> <li class="fragment">EOS + tapes...</li> <ul> <li class="fragment">EOS is CERN strategic storage platform</li> <li class="fragment">tape is the strategic long term archive medium</li> </ul> <li class="fragment">EOS + tapes = <span class="fragment" style="color:red;">&hearts;</span></li> <ul> <li class="fragment">Meet CTA: CERN Tape Archive</li> <li class="fragment">Streamline data paths, software and infrastructure</li> </ul> </ul> --- <h2>EOS+CTA <span class="fragment"><i style="color:blue;">Deployment</i></span></h2> ---- <!-- .slide: data-background="https://codimd.web.cern.ch/uploads/upload_d361eb4b4ad42029bd3d998a1600cfa0.png" data-background-size="70%" --> ---- <!-- .slide: data-background="https://codimd.web.cern.ch/uploads/upload_d2d164112f95cfd9fa22d4532281323e.png" data-background-size="70%" --> ---- <!-- .slide: data-background="https://codimd.web.cern.ch/uploads/upload_7d8fb723c75a802eb77a6e53037afe26.png" data-background-size="70%" --> --- <h2>EOS+CTA <span class="fragment"><i style="color:blue;">Architecture</i></span></h2> ---- <!-- .slide: data-background="https://codimd.web.cern.ch/uploads/upload_eac32c76dde5a45191434a90d54a4d5a.png" data-background-size="70%" --> --- <h2>EOS+CTA <span class="fragment"><i style="color:blue;">Timeline</i></span></h2> ---- <!-- .slide: data-background="https://codimd.web.cern.ch/uploads/upload_0ae96233cb49710754263e2d780a20b6.svg" data-background-size="100%" --> --- <h2>EOS+CTA <span class="fragment"><i style="color:blue;">Dev&oper</i></span></h2> <p class="fragment"> Tightly coupled software <span class="fragment">&rArr; <span style="color:red;">tightly coupled developments</span></span> </p> <p class="fragment"> <span class="fragment highlight-blue">Extensive and systematic testing is paramount to limit regressions<span> </p> <p class="fragment"> <span class="fragment highlight-blue">Extensive monitoring</span> in place to <span class="fragment highlight-blue">ease debugging</span> and <span class="fragment highlight-red">target high performance from day 1</span><span> </p> ---- <!-- .slide: data-background="https://codimd.web.cern.ch/uploads/upload_0e38a1afc20ff3b7ce635b01826a4b84.png" data-background-size="70%" --> ---- <!-- .slide: data-background="https://codimd.web.cern.ch/uploads/upload_c7b3e43b9d263e3d3c125f604b04801a.png" data-background-size="70%" --> ---- <!-- .slide: data-background="https://codimd.web.cern.ch/uploads/upload_1372053cb920f7ea8bb00ca001ab2f88.png" data-background-size="70%" --> --- ## CTA metadata stress tests **Small EOS instance VTL backed** <span><!-- .element: class="fragment" data-fragment-index="1" -->- Few retired 1Gb/s disk servers and a decent MGM for <span style="color: dodgerblue">EOS</span></span> <span><!-- .element: class="fragment" data-fragment-index="2" -->- A few VMs for the <span style="color: dodgerblue">CTA tape part</span></span> <span><!-- .element: class="fragment" data-fragment-index="3" --><b>Each VTL tape server exposes 20 virtual tape drives and can sustain <span style="color: crimson">>1kHz 1kB files</span></b></span> ---- ## CTA metadata stress tests <span class="fragment" data-fragment-index="1"><i style="color:blue;">Stage in</i></span> <span><!-- .element: class="fragment" data-fragment-index="2" -->Tests until CTA routinely archived <span style="color: dodgerblue">1M files</span></span> <span><!-- .element: class="fragment" data-fragment-index="3" -->- <b><span style="color: crimson">10M file test</span></b>:</span> <span><!-- .element: class="fragment" data-fragment-index="4" --><img src="https://codimd.web.cern.ch/uploads/upload_ab41de6dff6da58bff6df00f1e51e723.png" class="plain" width=80%></span> ---- ## CTA metadata stress tests <span class="fragment" data-fragment-index="1"><i style="color:blue;">Stage out</i></span> <span><!-- .element: class="fragment" data-fragment-index="2" -->Tests until CTA routinely retrieved <span style="color: dodgerblue">1M files</span></span> <span><!-- .element: class="fragment" data-fragment-index="3" -->- <b><span style="color: crimson">4M file test</span></b>:</span> <span><!-- .element: class="fragment" data-fragment-index="4" --><img src="https://codimd.web.cern.ch/uploads/upload_7a8ae04296e39c5d958f6afda70f3ec5.png" class="plain" width=80%></span> ---- ## CTA metadata stress tests <span class="fragment" data-fragment-index="1"><i style="color:blue;">CI</i></span> <span><!-- .element: class="fragment" data-fragment-index="4" --><img src="https://codimd.web.cern.ch/uploads/upload_0ee146a25b9fe3d26c47515d41ec0477.png" class="plain" width=80%></span> --- ## CTA HI Data challenge **Bigger EOS instance** <span><!-- .element: class="fragment" data-fragment-index="1" -->- 10x10Gb/s disk servers and a decent MGM for <span style="color: dodgerblue">EOS</span> (~2.4PB)</span> <span><!-- .element: class="fragment" data-fragment-index="2" -->- 14xT10KD tape drives for the <span style="color: dodgerblue">CTA tape part</span> (3.5GB/s)</span> ---- <!-- .slide: data-background="https://codimd.web.cern.ch/uploads/upload_62abf6f4de8bf484d242c443d22d0f45.png" data-background-size="80%" --> ---- ## CTA HI Data challenge <i style="color:blue;">Extended</i> <h2><span style="color:crimson;">Can we saturate the tape drive speed longer?</span></h2> ---- <!-- .slide: data-background="https://codimd.web.cern.ch/uploads/upload_aefab20fca3dc284c3a8402d23f29d8f.png" data-background-size="80%" --> --- # <span style="color: dodgerblue">CTA</span> VS <span style="color: crimson">experiment data transfers</span> ---- ## ATLAS stage in Several tests conducted with Atlas DDM team using Rucio and FTS. - 2 stage in tests of 200TB each - ~90k files of 2.6GB archived to tape - sub-optimal EOS instance (2 slow disk servers) ---- ## ATLAS stage in <img src="https://codimd.web.cern.ch/uploads/upload_dfa6cf2e22f47bff0ff9f705a6fbe419.png" class="plain"></span> <img src="https://codimd.web.cern.ch/uploads/upload_8d18a04f89dfd4626a3c073a48f6717e.png" class="plain"></span> ---- ## ATLAS stage out aka *Tape carousel* test took place during Hepix: - 3 x EOS disk servers (~3x260TB of raw JBOD space) - 6-10 x T10KD tape drives - 90k files retrieved from EOSCTAATLASPPS (tape) to EOSATLAS by rucio through FTS ---- ## ATLAS stage out <img src="https://codimd.web.cern.ch/uploads/upload_cdff0f357f4522aabad54db96a12de84.png" class="plain"></span> ---- ## ATLAS stage out <img src="https://codimd.web.cern.ch/uploads/upload_f08082d31f8d0839404ca282d05d7fa7.png" class="plain"></span> ---- ## ATLAS stage out DDM <img src="https://codimd.web.cern.ch/uploads/upload_5a6394a3c1efa419f01d3c548edbb60e.png" class="plain"></span> --- # <span style="color: dodgerblue">CTA</span> & <span style="color: crimson">LHC experiments</span> ---- ## <span style="color:crimson">ATLAS</span> - PPS instance is <span style="color:blue">up and running</span> - Will push a redundant 5% share of production into CTA ---- ## <span style="color:crimson">CMS</span> - PPS instance <span style="color:blue">is up and running</span> - Starting work with the Rucio team - no PhEDEx ---- ## <span style="color:crimson">ALICE</span> - ALICE and CERN IT already <span style="color:blue">moved CASTOR behind EOS</span> - During HI production DAQ went to EOSALICE first - Agreed to provide a CTA PPS instance ---- ## <span style="color:crimson">LHCb</span> - relies heavily on SRM - CTA does not provide SRM - SRMless transfers hiden behing GFAL2 - Agreed to write DAQ data to EOSLHCb instance first --- <!-- .slide: data-background="https://media.giphy.com/media/26BRqPg05olzXG1bi/giphy.gif" data-background-size="70%" --> --- <img src="https://codimd.web.cern.ch/uploads/upload_e393b27c3245bb4b8d8e327488433109.png" class="plain"></span>