1461 views
IT-CDA Service Documentation Strategy - Stages' Log === Most recent first: ## 2021/01/28 .docs.cern.ch sites in OpenShift today Count is 100: alcatel-admins.docs.cern.ch alice-flp-suite.docs.cern.ch aliops.docs.cern.ch ams-soc.docs.cern.ch applications-gateway.docs.cern.ch auth-admin.docs.cern.ch auth.docs.cern.ch authoring-admin.docs.cern.ch backup.docs.cern.ch bgi.docs.cern.ch cern-search-admin.docs.cern.ch cern-search.docs.cern.ch cern-search-internal.docs.cern.ch cernphone-admins.docs.cern.ch cernphone.docs.cern.ch cernphone-ios.docs.cern.ch cloud.docs.cern.ch cms-analysis.docs.cern.ch cms-b2g.docs.cern.ch cms-dmwm-test.docs.cern.ch cms-dmwm-wmcore-sphinx.docs.cern.ch cms-http-group.docs.cern.ch cms-intcom.docs.cern.ch cmssw-docker.docs.cern.ch cmstcds2.docs.cern.ch cmstcds2.docs.cern.ch codimd.docs.cern.ch collinearw.docs.cern.ch config.docs.cern.ch coral-cool.docs.cern.ch crg.docs.cern.ch daqling.docs.cern.ch devices.docs.cern.ch discourse.docs.cern.ch drupal.docs.cern.ch ecal-daq-docs.docs.cern.ch engineering.docs.cern.ch eoscta.docs.cern.ch eosops.docs.cern.ch faser.docs.cern.ch firia.docs.cern.ch fixed-telephony-service-admin.docs.cern.ch fixed-telephony-service.docs.cern.ch hl-bgv.docs.cern.ch how-to.docs.cern.ch icms.docs.cern.ch icms-support.docs.cern.ch indico.docs.cern.ch indico-ops.docs.cern.ch invenioils.docs.cern.ch inveniordm-qa.docs.cern.ch inveniordm.docs.cern.ch it-cda-ic-internal.docs.cern.ch it-e-learning.docs.cern.ch it-procurement-tenders.docs.cern.ch itk.docs.cern.ch jalien.docs.cern.ch jenkins.docs.cern.ch jira.docs.cern.ch kplatis-mycern-debug.docs.cern.ch latinos.docs.cern.ch lgad-milano.docs.cern.ch lbmonitoring.docs.cern.ch lpwan-admin.docs.cern.ch lpwan-service.docs.cern.ch mailservices.docs.cern.ch mdttp.docs.cern.ch mdttp-preview.docs.cern.ch mig-manager.docs.cern.ch mig-user.docs.cern.ch ml.docs.cern.ch mm-qa.docs.cern.ch mobile-admin.docs.cern.ch mobile-infra.docs.cern.ch my.docs.cern.ch mycern-rest-api.docs.cern.ch noted.docs.cern.ch noted-sdn.docs.cern.ch noted-tb.docs.cern.ch publishing-admin.docs.cern.ch publishing.docs.cern.ch raml.docs.cern.ch chiby.docs.cern.ch sentry-admin.docs.cern.ch sentry.docs.cern.ch service-management.docs.cern.ch simpleanalysis.docs.cern.ch slides-admin.docs.cern.ch slides.docs.cern.ch snow-newcomer-devs.docs.cern.ch student-bootstrap-mi.docs.cern.ch swan.docs.cern.ch tapeoperations.docs.cern.ch telecom-ops.docs.cern.ch tone.docs.cern.ch tso-b513.docs.cern.ch unimib-analyses.docs.cern.ch webeos.docs.cern.ch webframeworks.docs.cern.ch xwiki.docs.cern.ch ## 2021/01/19 meeting [Agenda](https://indico.cern.ch/event/983714/). Present: Maria, Michal, Pete, Pablo J., Andreas, Thomas, Eduardo, Tim. Apologies: Jean-Yves, Dimitra (end of contract). ### CodiMD archive Dimitra prepared [this report](https://codimd.web.cern.ch/s/ksEGCiXno#) for today before her contract ended, linked from the agenda. She returns in spring, not clear if she will continue working on this project. The project scope still covers _Public notes_ only. Reminder: In CodiMD public notes means Editable or Locked notes which are Published (as notes or slides). Today, a series of notes can be archived one after the other in the current UI. User page reload is needed. [JIRA issues](https://its.cern.ch/jira/secure/RapidBoard.jspa?rapidView=6837&view=planning.nodetail&epics=visible&issueLimit=100&selectedEpic=AUTHORING-1) are up to date. README file updated for newcomer developers. Discussions took place with Jean-Yves and Jose' about possible future integration with archived material they manage. Next steps: * Deploy the software in the production cluster. Michal with Caetan Tojeiro Carpente. * Put a _pilot_ banner on the entry page, not to raise expectations. * Michal to use Dimitra's API and implement a button to go from codimd to the archive. * Call users to try out https://test-codimd-archive.web.cern.ch/ (requires tunnelling through the CERN firewall), advertise to the CDA group first. * Record the feedback in [the JIRA CodiMD Archive Epic](https://its.cern.ch/jira/secure/RapidBoard.jspa?rapidView=6837&view=planning.nodetail&epics=visible&issueLimit=100&selectedEpic=AUTHORING-1) and do no action for now. * After codimd-cernbox integration we have to re-evaluate whether a separate CodiMD archive solution should be kept or storing in cernbox is enough. ### Twiki/xwiki/Foswiki * Pete prepared [these slides](https://indico.cern.ch/event/983714/contributions/4143564/attachments/2158461/3670747/DocumentationProjectFoswikiXWikiNewsJan2021.pdf) also linked from the agenda. * Many xwiki code changes in the last 2.5 months. Identified a few bugs and Pablo fixed them. They were mostly specific to our OpenShift set-up. * Foswiki core code is not changed since 2018. A lot of extensions added though... Many cern-made improvements in the area of e-groups' integration, search and performance. * There is no end of twiki support. * The reason of this investigation is that a recent users' Survey showed desire for modern features. * It seems better to make dedicated instances for the communities instead of a monolithic one. * xwiki offers an easier interface, better suited for sharepoint users. * both xwiki and foswiki are suitable for small and large groups. * importing twiki data into foswiki is seamless - going back also. This does not apply to xwiki. ### ODF Champions * The _Champions_ term was inspired by [this Microsoft programme](https://adoption.microsoft.com/become-a-champion/). * One person per IT group is nominated. Their mission is to collect their group's MS Office documents, test them, help with their migration. * Only 3 CDA members answered to Pete, our CDA Champion. CMF statistics show 30+ MS Office users in CDA. Therefore this number doesn't include MS Office documents of Mac users. * Pete to contact users (taken from the above statistics data) personally to insite them to reply. * The ODF Champions will meet **end of February** with the results of their inventory. ### A.O.B. Recommendations' reminder Following a question by Michal Kwiatek: 1. mkdocs: to be used for _Documentation_ pages, which should be published under web site names _myservice_**.docs**.cern.ch and _myservice-admin_**.docs**.cern.ch 2. SNOW SE, FE pages: continue to be updated by the service managers _in SNOW_. We didn't get an easy-to-use API, in order to automatically synchronise with any other info source. Important CDA service managers preferred this solution, anyway. 3. SNOW KB pages: remain the _choice_ of the service manager. Some services are in favour of _discourse_. Luckily, the web being based on Hypertext, there can still be one KB for such services, saying "Please visit our discourse site...". 4. IT Website: The Documentation Project was never approached for a proposal to a more attractive and up-to-date IT web site. We were told that an external company was hired to re-do it, together with other sites, in Drupal 8. 5. IT-CDA website: After trying for more than 2 years to convince CDA service managers to maintain their own pages, without success, the site can be considered _frozen_. It is still not very out-of-date, till the next IT restructuring... We did a lot of work for [this attractive list of CDA services](https://it-dep-cda.web.cern.ch/it-dep-cda/services/), so I still quote the site to candidate internship students. ### Next meeting Tuesday March 30th at 14h30. ## 2020/10/27 meeting [Agenda](https://indico.cern.ch/event/946542/) contains the slides related to the sections below. Present: Maria, Dimitra, Michal, Pete, Pablo J., Jose Benito, Tim, Andreas, Thomas, Jean-Yves, Eduardo. ### Slides The CERN web-based _Slides_ should be put on a _slow lane_ until the end of the year when the new IT plans and resources will take shape. We can give the link https://cern.ch/slides and the userguide https://slides.docs.cern.ch to individual interested contacts. It should be clear though, that _requests will be included in [the Slides' JIRA epic](https://its.cern.ch/jira/secure/RapidBoard.jspa?rapidView=6837&view=planning.nodetail&epics=visible&issueLimit=100&selectedEpic=AUTHORING-17) with no immediate action._ After sorting these JIRA tickets for _priority, required effort and timescale_, we can decide how to get help, for example with a new student, who will implement the fixes and improvements and update the admin doc https://slides-admin.docs.cern.ch. ### CodiMD archive Reminder: The project, by design, archives CodiMD _public_ pages, meaning _Editable_ or _Locked_ pages which are _published_ as _notes_ or _slides_. [Click here](https://test-codimd-archive.web.cern.ch/) to try the archiver. [See here](https://test-codimd-archive.web.cern.ch/s/S1r2wlnUQ/2020-10-26-12-11-43#) the _Documentation Project notes_ archived during this meeting. The **Indico** integration investigation was put on hold since our last meeting. The CodiMD archive development team, Dimitra and Michal, were busy thinking about the extension of the project scope to cover _Limited_, _Protected_ or even _Private_ notes. The latter require the owner to "open" the access rights, at least temporarily, for the archiving process to have access to the page. Indico integration could mean that Indico offers a button for archiving CodiMD notes uploaded as _Indico material_. Alternatively or in addition, Indico could use the _beta_ PDF export that CodiMD already offers to automatically generate and store a PDF file of the note. The CodiMD archive expansion to cover archiving .md files viewable from **CERNBox** was discussed. See [an example here](https://cbox-codimd-01:3000/T482RLobSqKG3jtSPpT5ag?metadata=https%3A%2F%2Fwopi.cernbox.cern.ch%3A8443%2Fwopi%2Ffiles%2F32594668%3Ft%3DeyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyaWQiOiIxMDA3MTU6MjY4OCIsImZpbGVuYW1lIjoiL2Vvcy91c2VyL3AvcGFibG9nby9TaGFyZVBvaW50L0NoYXJhY3RlcmlzdGljcyBvZiB0aGUgQWx0ZXJuYXRpdmVzIEV2YWx1YXRlZCBhbmQgRGlzY2FyZGVkLm1kIiwidXNlcm5hbWUiOiJkaW1vdSIsInZpZXdtb2RlIjoiVklFV19NT0RFX1JFQURfV1JJVEUiLCJmb2xkZXJ1cmwiOiJodHRwczovL2Nlcm5ib3guY2Vybi5jaC9pbmRleC5waHAvYXBwcy9maWxlcy8_ZGlyPS9fX215c2hhcmVzL1NoYXJlUG9pbnQgKGlkOjIxMTUyNSkiLCJleHAiOjE2MDQwNzAyODYsImVuZHBvaW50Ijoicm9vdDovL2Vvc2hvbWUtcC5jZXJuLmNoIn0.bX36-D958TaNuGSrBoUpKQey4BUpfPC1kzB7XcNaTAs&displayName=dimou#) - unbelievable URI! Other ideas during the discussion: The possibility to archive a note should be an **option _in_ CodiMD**. Archived notes, for preservation reasons, should be uploaded to CDS. Formal naming for the Document IDs should be studied, in order to also archive _Drafts_ in an official way. **ACTION** Dimitra and Michal to discuss offline with Jean-Yves. ### Twiki, Foswiki and xwiki Thanks to Pete and Pablo the Foswiki and xwiki instances are both in OpenShift now. The observation remains valid: _Foswiki is more suitable for large groups_, still this is not the only quality of Foswiki. It is also practical for group collaborations with many web sites. It definitely needs _evaluation by a larger group of people_ before we can make a _plan_ and a _roadmap_ for the future of _twiki_. Tim congratulated Pete for his excellent work. https://xwiki.docs.cern.ch, written by Pablo, is the _xwiki_ user guide with features and set-up instructions. Some of the features, e.g. _Calendar_ are not integrated with other CERN services, so they should be "hidden" in our instance. Pablo updated [the Decision Tree](https://cernbox.cern.ch/index.php/apps/files/?dir=/__myshares/SharePoint%20(id:211525)) to contain **only** recommended products and included, in the same directory, [the list of products rejected after evaluation](https://cbox-codimd-01:3000/T482RLobSqKG3jtSPpT5ag?metadata=https%3A%2F%2Fwopi.cernbox.cern.ch%3A8443%2Fwopi%2Ffiles%2F32594668%3Ft%3DeyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyaWQiOiIxMDA3MTU6MjY4OCIsImZpbGVuYW1lIjoiL2Vvcy91c2VyL3AvcGFibG9nby9TaGFyZVBvaW50L0NoYXJhY3RlcmlzdGljcyBvZiB0aGUgQWx0ZXJuYXRpdmVzIEV2YWx1YXRlZCBhbmQgRGlzY2FyZGVkLm1kIiwidXNlcm5hbWUiOiJkaW1vdSIsInZpZXdtb2RlIjoiVklFV19NT0RFX1JFQURfV1JJVEUiLCJmb2xkZXJ1cmwiOiJodHRwczovL2Nlcm5ib3guY2Vybi5jaC9pbmRleC5waHAvYXBwcy9maWxlcy8_ZGlyPS9fX215c2hhcmVzL1NoYXJlUG9pbnQgKGlkOjIxMTUyNSkiLCJleHAiOjE2MDQwNzg3NjYsImVuZHBvaW50Ijoicm9vdDovL2Vvc2hvbWUtcC5jZXJuLmNoIn0.-Hxq3DumHxQ76S0t5huJ3TYvAkhSwrQ47OsBE-5N2r4&displayName=dimou#), e.g. _Confluence_. Tim's recommendation is to do price assessments _before_ functional comparisons, not to spend time for products we can't afford anyway. In the case of _Confluence_, it offers a migration from _sharepoint_ at a license cost. Those who absolutely need it can purchase their own license. If we see a massive user demand, then we missed a community. In this case, we should check again the use cases and start again. ### Future of the project Maria came with the proposal to _suspend_ the formal and regular meetings of the project for now, based on the following reasons: 1. _sharepoint_ sites are left to die a natural death. If new sites are born, we are not informed to be influenced towards another solution. 2. _[The Decision Tree](https://cernbox.cern.ch/index.php/apps/files/?dir=/__myshares/SharePoint%20(id:211525))_ gets updated as a function of the products and the communities affected. They don't participate in the _Documentation project_. 3. The creation or migration of Markdown documentation based on https://how-to.docs.cern.ch is now understood and endorsed by those willing to join. Development extensions by the OpenShift experts are not foreseen. 4. The _CodiMD archive_ progresses and the developers can share reports at the White Area or ad-hoc meetings. 5. The _xwiki/foswiki_ investigation progresses at the pace of the twiki user communities' requirements. They will adopt the solution that matches their needs. The evaluators and deployers can share reports at the White Area or ad-hoc meetings. However, the CodiMD archive and Foswiki/xwiki discussions showed so much benefit from the exchanges between the Documentation Project members, that we decided to keep the project alive and organise a meeting in January. Maria will circulate a newdle for the date. ## 2020/08/12 meeting [Agenda](https://indico.cern.ch/event/930101/) contains all the full reports and links mentioned below. Present: Maria, Dimitra, Aristofanis, Pete, Pablo J., Jose Benito. ### Slides Summary: Points by Aristofanis and all the [details here](https://codimd.web.cern.ch/s/SyiR7Fxdr#Meeting-20200812) This is the last meeting with Aristofanis. Thanks and congratulations are due to him for this good work. Last November, when he started, the task seemed ennt rmous. ### CodiMD archive Status by Dimitra [here](https://indico.cern.ch/event/930101/contributions/3909695/note/) Reminder: In CodiMD _public_ notes means Editable or Locked notes which are Published (as notes or slides). José suggested: Very good the Indico integration. Important to study also CDS integration. ### Twiki, Foswiki and xwiki Report by Pete [here](https://indico.cern.ch/event/930101/contributions/3909696/note/) Recent investigation showed that Foswiki is suitable for very large groups of users but not only. The Foswiki and xwiki investigation doesn't entail a phase-out of _Twiki_. Maria suggested that Pablo updates [the Decision Tree](https://cernbox.cern.ch/index.php/apps/files/?dir=/__myshares/SharePoint%20(id%3A211525)&) to reflect: 1. the _abandoned Alfresco_ and 2. the _missing Overleaf_ 3. the _sharepoint online_ option (pay per user) 4. José suggested to consider (and represent in the _Decision Tree_) the archiving of old sharepoint sites in CDS, if they are of interest to the whole of CERN. Private such sites with owners gone belong to the CERN organisation. Therefore service admins and managers can have access for well justified purposes. Owners of private sites not updated for years can be asked. ### Next meeting October 20th at 3PM ## 2020/06/15 meeting [Agenda](https://indico.cern.ch/event/914208/) Present: Maria, Dimitra, Aristofanis, Pete, Emmanuel, Tim, Thomas, Pablo J., Eduardo, Andreas. ### Slides Aristofanis demonstrated https://cern.ch/slides at the [White Area](https://indico.cern.ch/event/870219/) last Monday. Input received at that meeting [here](https://codimd.web.cern.ch/s/SyiR7Fxdr#Feedback-from-the-20200608-White-Area). Questions for the operational era real soon now: 1. In which CDA section the application will be supported and maintained. 2. Which method should be used to attract users. 3. How many additional features we can afford to accept from the feedback/wishes received. 4. What if the open source community requires technical exchanges for the product evolution and we have no resources for these creative activities. 5. What if dependencies, pre-requisite packages and plugins move on and things break in the code. 6. What if security vulnerabilities arouse and nobody discovers them on time. 7. What if new browser versions create broken views. **Discussion** Tim congratulates Aristofanis for the achievement and for taking on board the feedback received. Open Sourcing can happen now, the project name to be decided. It will be included in [the CERN github space here](https://github.com/CERN). Not yet any _production service_ announcement. Integration with Indico (also maybe with [Phoenix](https://codimd.web.cern.ch/s/B1vMM3sEU#)) is vital for demonstrating its operational status. ### CodiMD archive Dimitra uploaded on the agenda the [actions done here](https://indico.cern.ch/event/914208/contributions/3844281/note/). **Discussion** Invite José Benito and Jean-Yves next time to share ideas and show the CodiMD archive implementation choices to them, for possible adoption for other archived material. Notifications' archive can also get good ideas from this work. ### Twiki, Foswiki and xwiki Pete showed [this presentation](https://indico.cern.ch/event/914208/contributions/3844280/attachments/2028363/3450180/DocumentationProjectFoswikiXWikiNews.pdf) on evaluation work he does with Pablo J. **Discussion** _Foswiki_ can be a good candidate for the thousands of ALTAS twiki pages, because the migration would be easy. _xwiki_ is a good _sharepoint_ alternative and also an Open Source alternative to _Confluence_. It is important to continue the tests and reach a recommendation soon. ### Next meeting **Wednesday August 12th @ 3PM** ## About the _.docs_ recommendation Discussion between Maria and Nikos. Input welcome from other Documentation Project members. In April 2020 we have 127 MkDocs sites in Openshift - see [list here](https://indico.cern.ch/event/890715/contributions/3756535/note/). Not all are _.docs_, as one can see... Given that the _.docs_ subdomain is only a _recommendation_ and these sites follow the [how-to guide](https://how-to.docs.cern.ch) anyway, the only way to present an incentive for the _.docs_ adoption is the creation of a _Directory of all Documentation sites_ and offer _a recipe for redirection_ from the _.web_ to the _.docs_ URL. The advantage would be: 1. Easy way for users to find a doc. 2. Documentation URL consistency across CERN. ## 2020/04/28 meeting [Agenda](https://indico.cern.ch/event/890715/) **Present:** Maria, Aristofanis, Nikos, Pete, Thomas, Eduardo, Pablo J., Michal, Emmanuel, Dimitra, Alex. ### Slides Aristofanis makes a demo of [the _slides_ application](https://cern.ch/slides) from his _localhost_ (authenticated access to the web site is not yet implemented). Immediate development items, their completion to be communicated to the project members **by May 7th**: 1. CephFS storage, then 2. new SSO integration. Long-term development is Phoenix integration for users to open _the slides application_ from within CERNBox/Phoenix. This depends on Phoenix, which is not ready yet. [See here the architecture alternatives](https://cernbox.cern.ch/index.php/apps/files/?dir=/__myshares/slides%20(id%3A242552)&). **Pending sine qua non functionality** 1. Slideshow 2. Export to PDF Maybe check the reveal.js engine to see how _slideshow_ is achieved. Given that the library already used already has slideshow mode, just hide the side bars. **Actions** 1. Unicode provision for multiple languages in the Presentation Title field and correct handling of escape characters (also in the filename the user gives when saving the presentation). 2. Check the unfolding of slides (maybe the apparent duplication in the unfolding is due to Zoom?) 3. Offer a button to position an image in the slide centre. 4. Allow _Save_ also via CTRL/S. Item already in JIRA. 5. Allow change of _theme_ for the whole of an existing given presentation, in any point of the editing process. 6. Show how each theme looks for the user to select. Maybe make a mosaic of the themes' layout for the user to select by clicking on it. 7. Allow setting an image as a background of a given slide (upload the image as _transparency_). **It is important for all to be able to play with the application from the web a.s.a.p., before giving more feedback!** ### CodiMD archive Nikos showed the CodiMD archive [CERN internal page](https://test-codimd-archive.web.cern.ch/) and summarised [yesterday's decisions](https://codimd.web.cern.ch/s/S1r2wlnUQ#20200427-CodiMD-archive-specific) following [the dedicated meeting with the Indico team](https://indico.cern.ch/event/909992/). **On the MkDocs to PDF conversion** * Dimitra is investigating the export of <service>.docs.cern.ch static documentation sites to PDF. Issues are met with relative links. * PDF export should not be part of the CodiMD archive project. If the CodiMD archive gives a frozen=persistent view of the document at a given time, no need for PDF. It is understandable that long-term preservation is the reason for converting to PDF today but there is loss of functionality in the Markdown-to-PDF conversion. Better make sure that nothing is missing from the CodiMD archived page in HTML. ### Twiki and Foswiki Pete summarised ATLAS feedback and Foswiki vs xwiki investigationhe did [since the last meeting](https://codimd.web.cern.ch/s/S1r2wlnUQ#Twiki-discussion). Conclusion is that _xwiki_ is best for _sharepoint_ users and _Foswiki_ is a smooth transition for _twiki_ users. Remember [the Decision Tree](https://cernbox.cern.ch/index.php/apps/files/?dir=/__myshares/SharePoint%20(id%3A211525)&)! Some details: 1. Twiki features will be lost in static MkDocs sites. The large users, ala ATLAS, won't be happy. 2. Pete is creating Foswiki sites in Openshift. The new SSO integration and ACLs require some work. The sites may be not just under one common umbrella, like twiki.cern.ch now. Multiple instances may be created, depending on the size of the user communities, e.g. one for ATLAS, a separate one for CMS etc. 3. Last year's twiki survey showed user _requirements for search and navigation_. Something has to change in this area, maybe _xwiki_ (possible recommendation for sharepoint) or _foswiki_. Pete will make one of each and share. 4. Eduardo suggests to focus on _xwiki_. This was also decided at the MALT technical coordination meeting. Pete agrees that **one** alternative should be recommended, not two. Foswiki has the advantage that it requires no learning curve for the users. This is why he is installing both to try with the same user data and conclude on a recommendation. ### Next meeting **Monday June 15th @ 3PM** [Agenda](https://indico.cern.ch/event/914208/). ## 2020/04/27 CodiMD archive specific [Agenda](https://indico.cern.ch/event/909992/) **Present:** Maria, Nikos, Pedro, Adrian, Michal K., Emmanuel, Dimitra. **Reminder of current design and set-up** (Nikos): * Only **public** CodiMD notes are archived so far, i.e. _Editable_ or _Locked_ notes which are _Published_ (as notes or slides). * A CodiMD _note_ is saved in the _archive_ => outside CodiMD and frozen in time. * User can create hisher own archive. * _Full search_ within one's archive is possible. Search is based on [MongoDB technology here](https://docs.mongodb.com/manual/text-search/#overview). * Storage is now done via a CephFS volume. * Archived notes have the _Edit_ button disabled, given that the note should not be altered. * They have a red ribbon on the right to show they are archived. The URL is the same with the timestamp appended. Number of _views_, _edit permissions_ (=protection level) are also visible _frozen to the time the archive was done_. * The archive is accessible (restricted access) via test-codimd-archive.web.cern.ch. The site is defined in Openshift. * User can see, on this site, the _history_ of each of his/her archived notes (the versions with their date/time of archiving). * The _html_ **and** the _Markdown_ format of the note are, both, stored in the archive. * There is a API of the archive tool. It is available for Indico integration. * Michal has access to the code. * All CERN logins have access to the archive. **Expansion of the scope required** Dimitra to open JIRA tickets in [the CodiMD JIRA epic](https://its.cern.ch/jira/secure/RapidBoard.jspa?rapidView=6837&view=planning.nodetail&epics=visible&issueLimit=100&selectedEpic=AUTHORING-1) on the following: 1. _Non-public_ notes should also be covered. 2. Make more user-friendly the _timestamp_ on the URL of an archived note. 3. Help the users understand which URL is expected for the _Published_ version of a note, when they submit it for archiving. 4. The _PersonID_ is passed now from the CodiMD archive API. This should be replaced by the _username_ to include non-CERN people (HEP users etc). Also because Indico doesn't have the _PersonID_. 5. Implement a download button for the note. 6. Offer a button in the archived page to _Save in my CERNBox_ (on demand). 7. After the CodiMD-Indico integration is done, Indico will archive the note automatically. The user will still be able to do it via the codimd-archive-web site. **Decisons and timescale**: * CodiMD upgrade will have to take place soon. The archive is a good safety mechanism to avoid things of the past breaking. * It is important that CodiMD slides of the past do **not** break with the upgrade, as we've been advertising them a lot in the MALT framework. * When archiving of _non-public_ notes will be possible as well, the Indico developers will start looking into the API for the integration. * Current statistics for CodiMD links on Indico: * startswith https://codimd.web.cern.ch/s/: 85 * startswith https://codimd.web.cern.ch/p/: 232 * startswith https://codimd.web.cern.ch/: 540 * Dimitra and Michal will discuss with Andreas about priorities and the timescale for the archive scope expansion. * MkDocs to PDF export will **not** be handled by this project. Dimitra is working on <service>.docs.cern.ch export to PDF. **Notes** by Maria with many thanks to Nikos and Dimitra for work done, Michal for taking over from Nikos, the Indico experts for accepting the integration, Emmanuel for input on our CodiMD plans and everyone for joining. ## 2020/02/18 meeting [Agenda](https://indico.cern.ch/event/883040/) Ismael, Pete, Thomas, Aristofanis, Pablo G-J, Dimitra, Nikos, Maria, Alex, Andreas, Eduardo. ### Slides Phoenix integration (new cernbox) in the coming days. Today's demo is done from Aristofanis's local host. When _saving_, the application makes a folder, captures the Redux state & prints it in a .JSON file, then adds the assets folder with the images, zips this folder and names it 'Presentation Title'.slides. The inverse happens when _uploading_ an existing presentation. **Action 20200218-1** [JIRA ticket](https://its.cern.ch/jira/browse/AUTHORING-163) for browsing mode to see the slides in full screen. **Action 20200218-2** [JIRA ticket](https://its.cern.ch/jira/browse/AUTHORING-43) to check how the current [CKEditor](https://ckeditor.com/) can be changed to achieve choice of _alignment_ and _font size_. Review the allowed options and check for various browsers. **Action 20200218-3** [JIRA ticket](https://its.cern.ch/jira/browse/AUTHORING-164) to allow background image change. Functionality-wise focus on: 1. font and alignment (editor) 2. background image 3. pdf export Phoenix integration is vital, of course, for users to try the application. ### CodiMD archive Full search done with elastic search. API ready for use from another application, e.g. Indico for a CodiMD page to be archived. Archiving can be done on a note basis or a user basis. When Indico archives on behalf of the user, the node will be read-only and cannot be deleted. **Action 20200218-4** Thomas will discuss a PDF version of the archive made in addition to the HTML one. Should it be made by Indico? **Maria organised a specific CodiMD archive meeting to answer this.** **Action 20200218-5** Maria asks if anyone sees a need for archiving non-public CodiMD notes. Nikos suggests to check Indico for codiMD pages linked as material and write down the technical feasibility of explanding the archiving to other notes. Pages which **were** _public_ (in the CodiMd sense, i.e. _Editable_ or _Locked_ notes which are _Published_ as notes or slides), hence archived and became _restricted_ later on, present an issue, i.e. what to do with such archived pages...**Maria organised a specific CodiMD archive meeting to answer this.** ### ASDF & ITUM feedback Similar feedback and very positive in both meetings. Suggestions were also very similar in the two fora. [Go here ](https://codimd.web.cern.ch/s/S1r2wlnUQ#20200217-ITUM-feedback)for all the details. **Eduardo** collected the following _Documentation requirements_ from Johannes Gutleber (CERN ATS-DO) #### CRITICAL FUNCTIONALITY (Required): - write documentation pages directly in a Web editor with formatting capabilities (bold, italics, headers, formatted text) - easily make tables (table designer rather than plain HTML) - embed images and videos in the text with page formatting (e.g. text flowing around the images) - versioning of the articles - page change history - comments for page changes - cross linking of the articles using a wiki style notation (not direct hyperlinks, but like in media wiki or Sharepoint using a functionality like [[page_title | page_name]] - Multi language support (the same documentation in English and French) - table of contents or navigation menu - add in on the same page with links to related information (e.g. related pages, related topics) - search functionality - tailoring of design (background, fonts, heading styles, footer contents, logo) - information managed in database with backup and restore capability #### ADDITIONAL FUNCTIONALITY (nice to have): - possibility to comment the documentation in a blog-style fashion - print and formatted print to PDF - PDF book creation of articles collections - public pages and pages with access restrictions (authenticated readers or role based access) - index creation ### Twiki discussion Twiki at CERN is actually quite complete. The only things I’d need advice on would be: - multiple language support for documentation in en + fr: find a solution (e.g. TopicTranslationsPlugin) - possibility to “skin” the project wiki to make it visually a bit more attractive / easier to ready. - For sectional editing support a plugin is needed. - Get a bit of help from a person who has a good experience with Twiki to set up the structure and recommendations on plugins. * twiki-like cross-references (active links on twiki names.) * multi-lingual design **Action 20200218-6** Pete to check with yesterday's ITUM participant Chris Lee (ATLAS) how they migrate twiki pages to MkDocs. **Pete's answer** _After contacting Chris Lee it appears that he was reflecting personal thoughts rather than those of Atlas. He clearly doesn't like using TWiki but this is not the feeling of Atlas on the whole. He moved some personal TWiki documents to MkDocs. I offered to help out but he has not replied to my 2nd email. For creating online documentation MkDocs and TWiki offer similar functionalities. MkDocs is for the creation of static web pages. However TWIki is a collabortion tool and offers far more than just for documentation and note-taking and in my view migrating 1000s of TWiki documents to MkDocs would not be a good idea - users would lose alot of the functionalities that they are used to with TWiki. It would be for the users to decide which topics are better of as static._ **Action 20200218-7** Twiki is migratable to [Foswiki](https://en.wikipedia.org/wiki/Foswiki). Pete will prepare a White Area lecture on this. **Pete's answer** _Foswiki is a fork of TWiki and has many similarities. It arguably has better support thank TWIki because of the number of developers. There are some nice features including a different look &feel and markdown support. Foswiki has been installed on Openshift and I have made an integration with Openid authentication and authorization using egroups thereby respecting the TWIki access controls. I have been looking at another option for TWiki - XWIki. XWiki is an open source product that has much in common with TWiki and also Confluence and Sharepoint. This option has also been discussed in our Sharepoint MAlt project. It can run on Openshift , with OpenID and with a dbondemand back-end. I am currently preparing 2 sites to compare with TWiki data imported. I will also then prepare my thoughts on the pros and cons of these options._ ### Documentation Project future The consensus was that the project is useful and should continue. Candidate investigation areas (not all scheduled to be done.) * The requested improvements of how-to.docs.cern.ch already in JIRA. * Markdown editor investigation. Part of Indico mandate or part of this group. (Thomas) * Implement a Web site type that simplifies the how-to instructions. (Alex) * Wordpress was publicised yesterday at ITUM. Not supported but available as is. (Andreas) * Support to drupal site owners in their transition or migration to Drupal 8. (Maria) Maria thanks Nikos for sharing the management of this project with her with technical expertise, collaborative and constructive spirit and for delivering the [valuable how-to document](https://how-to.docs.cern.ch). ### Next meeting **2020/04/28 @ 3PM in the Sprint Room** [Agenda](https://indico.cern.ch/event/890715/) ## 2020/02/17 ITUM feedback [Event agenda](https://indico.cern.ch/event/862894/) * Clarify, in such presentations, the _type_ of Documentation covered. * Make available more MkDocs' themes. For the rest it is brilliant! * Give an index of existing <service>.docs.cern.ch sites. * Expand the _search_ to cover more than the inernals of a given documentation. #### Answers by Nikos to questions raised at the ITUM Q: Which types of documentation is this targeted at – users, services, apps ? And what is the recommendation on how to structure the content? A: The project started for IT services documentation, both user-facing and internal. We suggest it is used for services outside the IT and for other contexts. Currently there are 36 documentation sites using the ".docs" subdomain. We don't have a recommendation for the structure of the content, users should organise it as it makes sense for them. Q: Is there a public directory of all documentation sites using this infrastructure? A: Not yet - but this is a good idea and it is planned for the future. It will be easier to implement following the future evolution of Web Services. Q: What about access from the TN? A: It may be preferable to access these docs using nodes connected to the GPN. Please bring this up at the CNIC meeting. Comment from ATLAS: We’ve started converting twiki’s to the how-to docs and it is a major improvement. Q: Is there a single search interface for all documentation sites? A: Not yet - but this is a good idea and it could be implemented with the upcoming new CERN Search Service. ## 2020/01/30 ASDF feedback [Event agenda](https://indico.cern.ch/event/880287/) [Minutes by the ASDF admins](https://indico.cern.ch/event/880287/contributions/3718211/note/) During the discussion on [Nikos's slides](https://indico.cern.ch/event/880287/contributions/3718211/attachments/1974918/3294420/2020-01-30_-_IT-ASDF__Data_transfer_challenge__documentation_project__update.pdf): * Nacho Barrientos says their documentation is migrated. Thanks! When moving from gitbook to MkDocs they lost the functionality of a [plugin which would be nice to have](https://www.npmjs.com/package/gitbook-plugin-terminal). He says they would love to have it again in MkDocs. Nikos says that maybe we could develop it at CERN and share it upstream with MkDocs. Or at least open it as an issue at the [MkDocs GitHub repository](https://github.com/mkdocs/mkdocs) for future implementation. The original plugin was developed by David Moreno in CERN IT. See [AUTHORING-148](https://its.cern.ch/jira/browse/AUTHORING-148). * Vincent Brillault asks about Editor recommendation. We looked at it and have no conclusion so far. Nikos says that an editor would be ideal if it could: * not complicate the production process * allow for less advanced users to write documentation * GitLab can be used to edit Markdown files directly, but it's not preferred by less advanced users. It could be possible that in the future that becomes more user-friendly and intuitive. * Tim Bell says that this has been a success given the amount of adopters within a few months. He argues that it is important to keep the process simple. He suggests to create a directory of all the \*.docs.cern.ch sites. Nikos says that: * this could be directly taken now from the OpenShift routes on production looking for `*.docs.cern.ch` (by the administrators). But this is not dynamic. * The CERN Search project is interested in indexing and searching the contents of all static documentation sites. * In the future Web Services infrastructure there should be a dedicated *Documentation* site type that would automatically identify all existing documentation sites. * See [AUTHORING-147](https://its.cern.ch/jira/browse/AUTHORING-147). * Jan Iven suggests a systematic dump into `.html` or `.pdf` format in order to have continuous access for end-users in case of OpenShift failure. The actual files are in GitLab. Nikos suggests that we could transpartently dump (`.html`) files on EOS in parallel to OpenShift. Tim Smith said that this will also be addressed as part of our business continuity policy. See [AUTHORING-146](https://its.cern.ch/jira/browse/AUTHORING-146). ## Status on 2020/01/20 Meeting [agenda here](https://indico.cern.ch/event/865409/). **Present**: Maria, Aristofanis, Eduardo, Andreas, Ismael, Dimitra, Nikos, Thomas, Alex. ### Slides Aristofanis showed codimd :-) [slides](https://codimd.web.cern.ch/p/S1tHwx7-L#/) about his development since our last meeting in November, based on priorities decided then. The _spectacle_ editor adaptation was very time-consuming. The very important and immediate need is to **find a way to run a _spectacle_ presentation project from within Aristofanis's _slides_ application**. The reason is to profit from in-built _spectacle_ functionality, like the pdf and html exports. It also gives a way to nicely store the presentations (folders and files architecture), which will be essential also for loading it. _iframe_ is an option, maybe not the best... (Eduardo). Another way is the use of _widgets_ (we used this in the CDA Jekyll site and in the careers.cern sites). Discussions on storage also started with Hugo and Piotr. The slides' hosting and accessing should preferably be through CERNbox. The banners of tools available while _making_ slides with Aristofanis's application https://cern.ch/slides were shown for comments. See image below. The recommendation is to solve the current technical problems, then have a look at _google slides_ (popular to CERN physicists, similar look to Microsoft Powerpoint) for sine qua non functionality. _The principle of our [slides](cern.ch/slides) application is to be web-based and functional without trying to substitute powerpoint._ ![](https://codimd.web.cern.ch/uploads/upload_a22b6d903fa4460171031c2cd472887a.png) **Action 20200120-1**: Aristofanis to check whether he needs _widgets_ or _iframe_ to integrate _spectacle_ in his application. Or which other API _spectacle_ supports that returns html. Eduardo will try to help, if needed. ### CodiMD archive Nikos presented [slides](https://codimd.web.cern.ch/p/HJ_bWx0gU) on: 1. the inception of the project (long-term preservation of notes that have a more "permanent" use case, i.e. presentations, minutes and even documentation), 2. the assumptions (e.g. a note can be accessed if its URL is known) and 3. considerations (e.g. full-text search & API for Indico integration). Dimitra presented the functionality (archive, view, history and user) and architecture (OpenShift, Nginx, flask, EOS, DBoD and *Elasticsearch*). The current assumption is that _public_ notes are archived, i.e. _Editable_ or _Locked_ notes which are _Published_ (as notes or slides). Archived notes will have the _Edit_ button disabled, given that the note should not be altered. Eduardo asks about the future of the rendering engine. Nikos says this project concentrates on long-term preservation of the content and will rely on other projects for the long-term preservation of rendering engines. Thomas says the Indico integration with the CodiMD Archive would very important for _Minutes_ and _Slides_. Possibly even directly through CodiMD as a default minute editor for Indico. Nikos says that replacing the Indico _Minutes'_ editor in Indico by codimd needs thorough consideration. **Action 20200120-2** Nikos and Dimitra to discuss with the Indico team about the API design. Alex said that for documents required to have versions, users use twiki. Eduardo (and Maria) say they use CodiMD, assuming the note will stay for ever. Aristofanis suggests a codimd button for the user to choose if archive is needed. Nikos says the underlying mechanism being now developed by the team would be needed anyway. In general, codimd is a pilot service at cern, it is based on hackmd and there is no assurance that the community will continue to maintain it. This is why the team avoids to make changes in codimd itself, while developing the archiver. Thomas asked precisions on the Indico API. ### Report on the Actions agreed in the 2019/11/18 meeting. * Action 1 Nikos to enhance the https://how-to.docs.cern.ch with the answers to technical questions received from IT members. They concern custom environments and advanced workflows which exist in other solutions: **Done** Alex, Iago and Nikos worked on the [Review before deploying](https://how-to.docs.cern.ch/advanced/review/). * Action 2 Nikos will check in OpenShift and Andreas will monitor via the WF section meeting progress on the section’s sites migration to <service>.docs.cern.ch [see screenshot](https://codimd.web.cern.ch/s/S1r2wlnUQ#Feedback-on-how-todocscernch): Alex counted the sites in **the .docs.cern.ch on OpenShift production are 36**. Andreas reminded in the WF section meeting for the other sites to be migrated. **Partially pending** * Action 3 Eduardo and Pete to complete the [Decision tree](https://cernbox.cern.ch/index.php/apps/files/?dir=/__myshares/SharePoint%20(id%3A211525)&) for Drupal, twiki and Documentation sites: Eduardo said that Pablo, Pete and himself walked through the Decision tree and eliminated some branches which are not reflected on the decision tree. Nikos asked about the status of _Wordpress_. Andreas said that a SNOW SE will be created emphasising that users are on their own. **Action 20200120-3** Pete, Pablo and Eduardo to reflect their new or edited branches in the Decions Tree on cernbox. * Action 4 Maria to post in the CDA mattermost channel the question to our python developers what they use to document. **Done** on 2019/11/25. Results on reStructuredText usage in CDA: * The Indico team still uses Sphinx/RST for technical docs and finds it quite good for technical documentation with links to code, etc… otherwise it’s over-complicated for general purpose usageIt’s only the user docs which is moved to MKDocs. * The same feedback was given by the codimd-archive developers (Nikos & Dimitra). Alex and team works on other requested applications to host on OpenShift to display static content, e.g. javascript single-page applications, a new type of web site we don't have at the moment. * Action 5 Aristofanis to understand the way cernbox authentication is done to open drawio files. **Done** on 2020/01/14. Result from a discussion with Giuseppe Lo Presti is that learning how the drawio integration to cernbox was done is useful and can be reused because the new project start is imminent and the APIs won't change too much. On the other hand there is no documentation, but Piotr knows well and he will share technical info. ### Documentation Project 2020 strategy Questions: 1. Mission in 2020 2. Priorities 3. Project management team (Nikos is leaving) * Andreas agrees with Alex that we have achieved stuff but there are new areas to be addressed, which have to be defined. Nikos will present the Doc. Proj. in ITUM (17th Feb) and ASDF (30th Jan), so we can get feedback from users. * Alex says that the technology for displaying static content and simplifying the way it is done is in the pipeline but not a priority. ### A.O.B. on SNOW KBs vs MkDocs FAQs Maria proposed, and everyone agreed, to close [the student project on SNOW KB conversion to .md files](https://it-student-projects.web.cern.ch/projects/documentation-project-fetch-snow-info-markdown-sites-api) for inserting them in <service>.docs.cern.ch sites. Reasons: 1. We can't oblige service managers to adopt a standard file hierarchy in their documentation sites, so that the KBs' content could be copied over _automatically_. 2. The main principle of the Web is based on _links_ ==> no content maintenance in >1 places. ### Next meeting **Tuesday February 18th at 3PM in the Sprint room** Agenda https://indico.cern.ch/event/883040/ ## Status on 2019/11/25 Twiki survey White Area By Pete Jones [see slides with all figures](https://indico.cern.ch/event/857140/attachments/1950682/3238223/TWikiSurveyResults.pdf). The amount of users who prefer _googledocs_ is impressive (700/1040). The collaborative editing feature is the one they probably like most. Most complaints were realted to _search_. Discussions with IT/ST showed that there will be an API (in spring?). They like _Confluence_ but we have to pay. We should ask them to try _xwiki_. Those (firemen and some admin people for high management) who use _sharepoint_ for their workflows, cannot be served by MarkDown or Twiki or Confluence or alfresco or foswiki or xwiki. foswiki a clone and competitor of twiki has more developers but twiki has the original guys who still keep it alive and working. Because of the very different use cases we have, obviously one solution won't cover all of them. Now we have a list of products we propose. We should now bridge between them. ## Status on 2019/11/18 Meeting [agenda here](https://indico.cern.ch/event/854786/). Present: Tim S., Ismael P., Andreas W., Eduardo A., Nikos K., Maria D., Aristofanis Ch., Thomas B., Pablo J., Michal K. ### About MkDocs sites and how-to.docs More dithyrambic comments were received from IT ASDF members. See [here](https://codimd.web.cern.ch/J40rhuevQAqQ12oqNtgZ-g?both#Comments-on-the-how-todocscernch-from-IT). Requests for clarifications and f2f _coaching_ for following the https://how-to.docs.cern.ch instructions make it obvious that we need to foresee some amount of _support effort_ in this project. **Action 1** Nikos to enhance the https://how-to.docs.cern.ch with the answers to technical questions received from IT members. They concern custom environments and advanced workflows which exist in other solutions. All development items are recorded in [the AUTHORING Jira tracker](https://its.cern.ch/jira/projects/AUTHORING/issues/AUTHORING-4?filter=allopenissues). **Action 2** Nikos will check in OpenShift and Andreas will monitor via the WF section meeting progress on the section's sites [see screenshot here](https://codimd.web.cern.ch/s/S1r2wlnUQ#Feedback-on-how-todocscernch). ### About sharepoint and other formats Following discussions with Bruno, Pablo and more [-see notes here-](https://codimd.web.cern.ch/s/S1r2wlnUQ#Sharepoint) we decided that sharepoint sites are old, few, private and _should be migrated according to Pablo's recommendations or just moved to cernbox as they are or be left to die a natural death_ rather than put effort in writing scripts that map .asp to .html and/or .md. Tim asks about existing sharepoint documentation on Windows and other CDA services. Michal said that it should be re-written. This is what happened with mail services, re-written from https://mmmservices.web.cern.ch/mmmservices/ to https://mailservices.docs.cern.ch/. **Action 3** Eduardo and Pete to complete [the Decision tree for Drupal, twiki and Documentation sites](https://cernbox.cern.ch/index.php/apps/files/?dir=/__myshares/SharePoint%20(id%3A211525)&). Eduardo presented his slides for the LHCb workshop today [here](https://codimd.web.cern.ch/p/BkqpuNJnH#/). Feedback he received: - Concerns about global search (protected data) - reStructuredText currently widely used - Codimd use case, improve archival/long term storage/indico integration for presentations - Unclear best option when need to protect documentation. Discussion on the feedback from the LHCb workshop presentation [summary slide here](https://codimd.web.cern.ch/p/BkqpuNJnH#/16): 1. They don't like to migrate [reStructuredText](https://en.wikipedia.org/wiki/ReStructuredText) to MkDocs. One of the reasons is that RST is well integrated into github and gitlab. Also, Python creates RST automatically. Indeed, so far, we recommend MkDocs for _User Documentation_. We should investigate this new use case. Nevertheless Indico was also Sphinx and was migrated to gitbook and then to MkDocs. So we do have the proof that a successful migration was done. **Action 4** Maria to post in the CDA mattermost channel the question to our python developers what they use to document. [**Done** on 2019/11/25](https://mattermost.web.cern.ch/it-dep/pl/qa8kqk1b7bd1tpy7c69r877sdy). Results: * The Indico team still uses Sphinx/RST for technical docs and finds it quite good for technical documentation with links to code, etc... otherwise it's over-complicated for general purpose usageIt's only the user docs which is moved to MKDocs. * The same feedback was given by the codimd-archive developers (Nikos & Dimitra). 2. LHCb asked about _search_. We don't have a solution yet, working on this. 4.codimd archive was asked by LHCb as well. Nikos and his technical student Dimitra work on that. A solution is expected 1st quarter 2020. ### Slides Aristofanis explained the choice of [_spectacle_](https://github.com/FormidableLabs/spectacle) as a basis for his development of _an in-house, web-based slides' editor_. He went through the demo of https://cern.ch/slides. So far he stores the state of the slide, as html fies in a DB. Please [see here discussions with Michal and Giuseppe](https://codimd.web.cern.ch/npmeRtJkTluBvdNNh3b42g#20191113) on cernbox API development and storage solutions. If the file will appear on cernbox as _filename.slides_, we should see how to invoke the way spectacle uses to export as pdf. Or rely on Indico for the pdf export. _spectacle_ was preferred to _reveal_ because it allows the developer to use REACT components to write the slides' editor. Pablo asks how many tools to make slides we shall have. Tim explains that .pptx is a thing used so far. If https://cern.ch/slides is as attractive in production as it looks at its first steps, then the codimd slides' module will be abandoned. **Action 5** Aristofanis to understand the way cernbox authentication is done to open _drawio_ files. Title/text/image editing and automatic rendering is very easy on codimd slides. So Thomas recommends to concentrate on providing this functionality before moving the text box around. Header, Footer, Master slide and 2-3 templates of text boxes are good enough to start with. ### Next meeting Monday 20th January 2020 Agenda https://indico.cern.ch/event/865409/ ## Preparing for [the 2019/11/18 meeting](https://indico.cern.ch/event/854786/) Notes from the discussion on 2019/10/22 (Aristofanis, Bruno, Eduardo, Maria D., Nikos, Pablo J., Pete). **Action 1** [This chart](https://cernbox.cern.ch/index.php/apps/files/?dir=/__myshares/\ (id:211525)) on Pablo's original will cover the flows for Drupal, twiki and Sharepoint. Of course, even within a given use case, e.g. _survey_ confidentiality of data or user's expertise, may lead to the adoption of a different tool. ### Drupal A motivation for some users to evaluate another solution is the Drupal 7 to 8 migration. Analysis by Eduardo: Thinking about the migration from Drupal 7 into the MkDocs solution, I've run some analysis over existing sites to check how many of them are actively using Book module which implements similar hierarchy documentation functionality. The results are that only 12 official sites are actively using it, see list https://lhcb-shifters.web.cern.ch https://alipub.web.cern.ch https://biodynamo.web.cern.ch/ https://alice-daq.web.cern.ch/ https://alice.web.cern.ch/ https://root.cern/ https://quality.web.cern.ch/ https://cms.cern/ (public website) https://cms.web.cern.ch (collaboration website) http://information-technology.web.cern.ch/ https://procurement.web.cern.ch https://moedal.web.cern.ch/ We could start by simplifying the http://information-technology.web.cern.ch/. All it needs is to become a list of Documentation sites, one for each service. Most Drupal sites will be migrated from 7 to 8. Few of the owners, with static pages did or will migrate to markdown. If we offer Wordpress on OpenShift, some will be encouraged to use it instead of Drupal for easy use cases. Making Wordpress a supported service is also being evaluated. ### Twiki Survey results' presentation planned by Pete [White Area 2019/11/25](https://indico.cern.ch/event/857140/). This will give us ideas on the existing use cases. Pete is evaluating _foswiki_. It is possible that he'll provide conversion tools. _xwiki_ may follow in his evaluation efforts. ### Sharepoint The mattermost channel for Sharepoint discussions is [_MALT-WorkSpaces_](https://mattermost.web.cern.ch/it-dep/channels/malt-workspaces). The June 2019 [survey](https://cernbox.cern.ch/index.php/s/fuYkdEbbCjPk1Aa) showed that users like most in sharepoint the collaborative editing and doc contribution. [Recommended Sharepoint Alternatives document](https://codimd.web.cern.ch/qg_5ahNzSHeohOsbLWzx3A) (Pablo, Eduardo, Pete). They should be offered to users, based on their use case. The list shouldn't be vast, so the choice will be easier: 1. [Confluence](https://codimd.web.cern.ch/qg_5ahNzSHeohOsbLWzx3A#Confluence) - licensed - free **if** the project is Open Source. Zenodo is a use case. BE/CO use a customised version of Confluence and paid for it. Data export is tricky because it is not stored in markup or markdown. No CERN SSO integrated out of the box. There is a plugin we could activate. 2. [Alfresco](https://codimd.web.cern.ch/qg_5ahNzSHeohOsbLWzx3A#Alfresco) - free. 3. [WordPress](https://codimd.web.cern.ch/qg_5ahNzSHeohOsbLWzx3A#WordPress-) - free (CMS) - Data export is tricky because it is not stored in markup or markdown. 4. [XWiki](https://codimd.web.cern.ch/qg_5ahNzSHeohOsbLWzx3A#XWiki-) - free (ok for project management). 5. Jira - site license exists - Recommended for small/medium project management - [See related White Area lecture](https://indico.cern.ch/event/827885/). 6. [Drupal](https://codimd.web.cern.ch/qg_5ahNzSHeohOsbLWzx3A#Drupal-) - free - CMS & powerful survey tool. 7. [FosWiki](https://codimd.web.cern.ch/qg_5ahNzSHeohOsbLWzx3A#Foswiki-) - being evaluated by Pete this year. 8. cernbox - official strategic policy - free. Documents on other solutions: * https://sharepoint-cloud-docs.web.cern.ch/sharepoint-cloud-docs/ (with MAlt alternatives' section) * https://codimd.web.cern.ch/KN0CXUUrS2eJnGInVqUFFQ# (Office 365 for those who need to stay with MS solutions...). Those who absolutely need to stay in sharepoint, will use this option. There is no migration path to this solution. A new site should be defined and content must be ported. This is not automatic. The license cost per user is might be cheaper than the one for Confluence. Nevertheless, it is not our role to raise the cost issue with the users. **Action 2** Make a website per platform and/or a video that explains to the user what functionality can be expected from each. Then contact the sharepoint sites' content owners. ### Progress on the [Action from 2019/10/07](https://codimd.web.cern.ch/s/S1r2wlnUQ#Sharepoint-migration-and-Documentation-Project) Pablo extracted sharepoint sites which possibly contain Documentation (URL, owner, dates of last visit and last update). They can be found in https://cernbox.cern.ch/index.php/s/RRK1zWOwceHw9kv. They correspond to site types: Wiki Wiki Pages Wiki Page Library * The value _Last visit_ is not possible to get, because the audit log is disabled. * These sites in some cases are not only used for documentation but also for collaborative work or as a document repository. The latter can easily be recommended to move to cernbox, if the datea there are still valid. * Bruno recommended _against_ opening the sites (in the cernbox link above) without owner's approval, even if they date since 2011, as they may contain confidential info. * Not sure it is a good idea to migrate the sharepoint sites to MkDocs. They seem to be 8-years-old on average, maybe effort should be concentrated in new documentation creation and these sites should be left to die...If we decide to migrate, in order to automate the process, the SharePoint API is web based and thus accessible from any OS: * [Documentation page about how to authenticate to consume the SharePoint API](https://espace.cern.ch/it-dep-is-internals/Sharepoint/Sharepoint2010/Useful%20info/Programmatic%20access%20to%20SharePoint%20services.aspx). * [Documentation on the SharePoint client-side API](https://docs.microsoft.com/en-us/sharepoint/dev/sp-add-ins/complete-basic-operations-using-sharepoint-client-library-code). * The respective list wiki library should be then used for each page to get the property item that contains the text in html format. * html-to-md conversion should be done separately. Aristofanis tried on _ubuntu_ and found only _.net core_ available up to API v.3. For _.net framework_ the available version is up to v.4.5. The libraries of Microsoft Sharepoint client require a higher version. **IF** we want to extract and convert, should we: 1. try from another OS? 2. extract from the server side? ## Status on 2019/10/07 Present: Michal, Alex, Emmanuel, Pete, Pablo (Jimenez), Thomas, Maria (Dimou), Aristofanis, Nikos (on vidyo). Apologies: Tim, Andreas, Bruno. This meeting had an [Indico agenda](https://indico.cern.ch/event/853420/). Check the presentation material there. ### Feedback on how-to.docs.cern.ch The feedback from IT ASDF members was -so far- "Congratulations" (email sent on Oct 3rd). See [all details here](https://codimd.web.cern.ch/J40rhuevQAqQ12oqNtgZ-g?both#Comments-on-the-how-todocscernch-from-IT). It is now time to move all CDA internal and/or user-facing documentation to <service>.docs.cern.ch or <service-admin>.docs.cern.ch (MkDocs, gitlab,OpenShift). The list of WF internal documentation with variable naming conventions is appended here. **Action Andreas:** Make individual migration plans at the WF section meeting. ![](https://codimd.web.cern.ch/uploads/upload_024e105d6f3b0ecee12770955b55ed4c.jpg) Michal agreed to go ahead with AD documentation as well. He asked for a recipe for documentation portions that have to appear in multiple places. Nikos documented this [in the how-to Advanced Topics](https://how-to.docs.cern.ch/advanced/custom_python_packages/). This is a topic about adding custom Python packages to MkDocs, including a Python plugin that serves the purpose of embedding snippets in more than one locations. ### Sharepoint migration and Documentation Project This discussion defines part of [Aristofanis's _Main project_](https://it-student-projects.web.cern.ch/projects/malt-related-project-standard-documentation-workflow-and-conversion-tools-documentation). Despite the fact that it is hard to identify _which_ of the thousands of sharepoint sites contain documentation pages, the need to _cover all use cases_ and give clear recommendations to the users led to the following agreement: **Action Pablo, with help from Bruno:** List the sharepoint sites that might be documentation (e.g. check use of the wiki libraries) with: 1. Site URL 2. Content owner's name 3. Last update date 4. Last visit (from the logs) This list will help us with the analysis: * quantity of sites to migrate, * likely up-to-dateness of the content, * applicability of a MkDocs solution, * need -or not- and effort for a conversion tool, * migration effort **Action CDA members:** Please send us your own sharepoint sites containing documentation pages. See [Maria's mattermost post of September 26th in the CDA service managers' channel](https://mattermost.web.cern.ch/it-dep/pl/yeeienhza3rkte3rtthu8jqafw). Thomas mentioned [the use case of FAQs in sharepoint](https://espace.cern.ch/AVC-workspace/default.aspx) (maybe not accessible by all CDA) which would profit from a conversion tool to MkDocs, for the service managers to integrate them directly in their documentation pages **(Action Aristofanis)**. html-to-mkdocs conversion would also be useful, not for MALT but for page formating purposes. ### Non-powerpoint slides This is the 2nd part of Aristofanis's work and leads to his BSc thesis. [This recipe](https://twiki.cern.ch/Edutech/NonPowerPointSlides) is insufficient for users who don't want to use Markdown syntax to make their slides. Aristofanis presented [alternative products](https://slides.com/aristofanischionis/first-test#/) and [a short video](https://www.youtube.com/watch?v=nB9Wvb2g6Sw) on how to use https://slides.com/, a recent company, co-founded by the author of _reveal_ [see here the CERN instance](https://reveal.web.cern.ch/reveal/). Our policy is to replace Microsoft PowerPoint with an _open source, web-based,_ (ideally) _free of charge_ product that covers our use cases with a different philosophy than the ".pptx substitutes" à la LibreOffice, OpenOffice etc. We also don't wish to resort to Google. Hence: **Action Emmanuel:** Discuss with Tim whether a negotiation with slides.com is a good idea. The argument would be based on a _collaboration_ rather than a good license price. **Action Aristofanis:** Make a prototype of a CERN-made slide-maker based on [reveal](https://www.w3.org/2013/06/revealjs/#/). Timescale and maintenance concerns were discussed. If the Open Source community endorses the tool and helps, this effort might give an ideal outcome. ### ATLAS request on CDA Documentation recommendations Maria was asked to present again IT Documentation recommendations to an ATLAS Workshop in December [(see here last presentation)](https://dimou.web.cern.ch/dimou/reveal/reveal.js/atlas-20171211.html). We agreed to the following actions: **Action Pete:** Thanks to his current survey to make an inventory of twiki use by ATLAS. **Action Maria, Emmanuel:** Prepare together and present a global view of CDA/MALT recommendation covering multiple tools, including drupal. Input by all doc. proj. members is indispensable. ### Next meeting Monday 18th November @ 3PM CET **in room 28-R-015**. Please see event https://indico.cern.ch/event/854786/ to prepare. ## Comments on the how-to.docs.cern.ch from IT ``` On 25.10.19 10:11, Coralie wrote: > Hello Maria, Hello Nikos, > > Thank you so much for the excellent guide on how to produce documentation with MkDocs. > I'm following the guide right now, and I must say, I've rarely seen such a > well-written how-to! > Thanks a lot for the effort you put into this, much appreciated. > Cheers, > > Coralie (a technical student from IT-CS-CE documenting her code right now) On 04.10.19 11:25, Lionel Cons wrote: > CERN Authoring - Documentation writes: > > We now invite you to visit the how-to guide we have prepared: [...] > > I have used your guide to create two docs web sites for the messaging services. > > Your guide is very well written and everything works as expected. > > Congratulations. > > Cheers, > > Lionel On 04.10.19 09:31, Sebastian Lopienski wrote: > Dear Maria, Nikos, > > Congratulations for completing this project, AND for preparing such an excellent how-to documentation. It is a real pleasure to read, and so clear!! > > I’m actually looking forward to deploying some documentation as soon as I need to do so! :-) > > Cheers, > Sebastian > > >> On 3 Oct 2019, at 12:36, documentation-contact (CERN Authoring - Documentation) <documentation-contact@cern.ch> wrote: >> >> Dear IT-ASDF attendee, >> >> At the IT-ASDF event on March 14th 2019 we presented our plans for a common documentation workflow, followed by an interesting discussion. >> >> We now invite you to visit the how-to guide we have prepared: <https://how-to.docs.cern.ch>, create or migrate your Markdown-based static documentation site with MkDocs, adopting the <service>.docs.cern.ch convention, and send us your feedback and comments. >> >> If you already have documentation with GitBook you might find this specific section on migrating to MkDocs useful. If on top of that your current documentation with GitBook is hosted on WebEOS, you might find this advanced topic also useful. >> >> Thank you and best regards, >> >> Nikos Kasioumis & Maria Dimou ``` ## Status on 2019/09/30 Notes from the [White Area talk for CDA members by Nikos](https://indico.cern.ch/event/829649/) navigating through the https://how-to.docs.cern.ch/ Instructions' site, on **How to create or migrate documentation to MkDocs/OpenShift**. Clarifications based on audience questions: 1. Yes, a separate site should be defined for every documentation (question by Pete). 2. URL customisation (Step 5 in the "New Documentation" section) is about how to adopt the .docs subdomain, as opposed to the default .web. This is not mandatory but clearly recommended by the Documentation Project. 3. A command-line operation or web form to set-up and sync between web services, gitlab and OpenShift was suggested by Pedro, to save time from following expicitely the multiple steps listed in https://how-to.docs.cern.ch. We estimate this 'nice-to-have' feature of low priority because of variable expertise level in our user community and non-negligeable maintenance cost of such script/form as well as other development priorities in the Documentation Project. 4. In the [Advanced topics](https://how-to.docs.cern.ch/advanced/): The instructions on how to [customise the Nginx web service](https://how-to.docs.cern.ch/advanced/nginx/) to unambiguously impose the site name (Redirection) were explained. 5. Restricted (e.g. Admin) sites should be different from the public site. E.g. Thomas' sites are https://fixed-telephony-service.docs.cern.ch/ and https://fixed-telephony-service-admin.docs.cern.ch/. Alex confirmed this is the best way to go, i.e. keep things separate (public vs restricted). **Action** Nikos will include this additional recommendation (about service-admin.docs.cern.ch naming convention) to the "Keep in mind" section of https://how-to.docs.cern.ch/new/new_site/. 6. Tim asked what about other formats, not only migration out of gitbook. Maria said that Sharepoint .aspx to .md file conversion is part of [Aristofanis's project](https://it-student-projects.web.cern.ch/projects/malt-related-project-standard-documentation-workflow-and-conversion-tools-documentation), starting tomorrow. Tim said that the Drupal service should suggest to their users how and when their sites could/should be migrated. Eduardo said that some Drupal sites already migrated to MkDocs but not all of them are static and massive adoption will be difficult. **Action** Nikos will create an _Advanced Topic_ inhttps://how-to.docs.cern.ch, called _Basics and Best Practices on how structure a MkDocs documentation_. Pete asked about plans for twiki to Markdown migration tools/script/plans. This _is_ also part of [Aristofanis's project](https://it-student-projects.web.cern.ch/projects/malt-related-project-standard-documentation-workflow-and-conversion-tools-documentation) with lower priority, compared to purely Malt activities (.aspx and .pptx alternatives). **Action** For every format to MkDocs migration we should document for the users possible gain or loss of functionality e.g. [twiki tables](https://twiki.cern.ch/LCG/MiddlewareReadiness#Baseline_Versions) vs [Mkdocs tables](https://codimd.web.cern.ch/Sfmu0gJARYeYvBZNYe0gCw#). 8. The automatic URL migration in a current working item in Alex's team e.g. currently https://mailservices-docs.web.cern.ch/ and https://mailservices.docs.cern.ch/ co-exist and display identical content. **Action** Nikos to check and include in https://how-to.docs.cern.ch/. ## Status on 2019/08/19 **Present:** Tim, Emmanuel, Andreas, Alex, Nikos, Maria, Aristofanis. 1. [Instructions](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#) are now a proper MarkDown site: <https://how-to.docs.cern.ch/>. Thank you Niko! Points from the discussion: 1. Nikos will now configure OpenShift to display the specific URL https://how-to-docs.web.cern.ch/ as https://how-to.docs.cern.ch. 2. The site name _<service>-docs_ will be explicitely required in this _how-to_ documentation. 3. The same documentation doesn't yet explain the way to turn a URL from <service>-docs.web.cern.ch into <service>.docs.cern.ch. This will be added now. 4. The way to register a _restricted access'_ site, will also be added in the same _how-to_ documentation, even if this part will have to be re-written later. For now, restrictions are only possible for **separate** web sites, not sub-directories. As the readers and adopters will be ASDF members from IT dept, we expect no problem in understanding the process. 5. Nikos got positive feedback from BE dept. They ask what to do with members of their projects who have no CERN login, hence they can't publish gitlab.cern.ch. Alex suggest that they do follow the _how-to_ instructions, while continuing to publish their doc in github.com or gitlab.com. When Federation is implemented in the new AuTH model, we may give them gitlab.cern.ch access. The new model will contain Roles and Groups. This allows "account" grouping under CERN, FNAL etc and facilitates accounting for Virtual Bills. Eventually, there will be some form of real charging because we pay for gitlab. 6. The re-direct implementation and sub-domain subscription are still pending in the IT-CDA-WF developers' team. 2. Welcome to Aristofanis who will start in a month to work on _conversion tools_ for documentation (other formats and/or dynamic pages), slides and other [MAlt needs](https://indico.cern.ch/event/799512/). See here [Aristofanis's project description](https://it-student-projects.web.cern.ch/projects/malt-related-project-standard-documentation-workflow-and-conversion-tools-documentation). This project description was discussed. The idea is that the _Main Project_ part on MAlt conversion tools for sharepoint etc documentation and off-the-shelf alternatives to .pptx slides will the Aristofanis's MSc thesis. COnsensus was that development contribution to the gitlab/OpenShift/AuTH _infrastructure_ should be left as last priority for him. 3. **Next meeting: Monday 7th October @ 3PM in the Social Room**. ## Status on 2019/07/31 Rescheduling Documentation project checkpoint meeting of August 1st due to clash with ASDF talk by Daniel on _the latest gitlab CI setup_. New date **August 19th @ 3 PM.** Invitation sent. Unfortunately Thomas won't be back yet but notes will be https://codimd.web.cern.ch/s/S1r2wlnUQ By that time the emails to IT members of the ASDF will have been sent and feedback collected. ## Status on 2019/07/18 a. Thanks to Nikos, EOS-defined documentation sites http://fixed-telephony-service.web.cern.ch/ and https://it-e-learning.web.cern.ch are now migrated to OpenShift and to the new naming scheme http://fixed-telephony-service.docs.cern.ch/ and https://it-e-learning.docs.cern.ch and _Redirects_ are set. b. The sub-site https://it-e-learning.docs.cern.ch/administration/, which had restricted access in EOS, is now public, until SSO access is ready. c. We couldn't find the admin web site for the fixed-telephony-service corresponding to https://it-e-learning.docs.cern.ch/administration/. Where is it? d. Nikos enhanced the [Instructions](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#) and prepares a proper MkDocs site. Please comment on the site name proposals https://howto.docs.cern.ch (by Nikos)or https://documentation-instructions.docs.cern.ch (Maria)or https://howto_write_doc.docs.cern.ch (by Maria) or _what else?_ e. Nikos will send individual emails to the ASDF members from IT **during the 1st week of August** (he has the individual emails' list). See [here the exact text of the mail, which should be edited to replace the Instructions' URL when the proper site, with name decided in d above, is ready!](https://codimd.web.cern.ch/s/S1r2wlnUQ#Actions-decided-on-20190522). Known dates of absences of the documentation-project management team: 20-31/7 Nikos 23/8-14/9 Maria 14/9-30/9 Nikos (CSC) f. Technical student Aristofanis Chionis will work, as of this autumn, on conversion scripts for other documentation formats and MAlt tools, mainly for making slides. Please [see the project description here](https://it-student-projects.web.cern.ch/projects/malt-related-project-standard-documentation-workflow-and-conversion-tools-documentation) and comment. g. Checkpoint meeting **August 1st at 4pm in the social room**. Please reply to the invitation! Thank you! ## Status on 2019/06/19 a. Alex deployed in production the code for <service>.docs.cern.ch. The OpenShift-defined site _mailservices-docs.web.cern.ch_ is now migrated to _mailservices.docs.cern.ch_. This is very good timing in view of ITMM & ITTF Kopano presentations next week, where our recommended technology and naming scheme for documentation can also be mentioned. _Redirect_ from _mailservices-docs.web.cern.ch_ to _mailservices.docs.cern.ch_ is not needed because the files are the same in OpenShift, only the displayed URL changes. When SSO access is implemented, we should re-check this. b. EOS-defined documentation sites https://it-e-learning.web.cern.ch/ and http://fixed-telephony-service.web.cern.ch/ will be migrated to OpenShift and then to the new naming scheme https://it-e-learning.docs.cern.ch/ and http://fixed-telephony-service.docs.cern.ch/ by Maria and Nikos mid-July, in order to also enhance the [Instructions](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#). When this migration will be done, _Redirect_ shall be needed. c. As agreed on June 3rd (see earlier notes), the individual emails to the ASDF members from IT will be sent **during the week of June 24th** by Nikos (currently on holiday but he has the individual emails' list). See [here the exact text of the mail](https://codimd.web.cern.ch/s/S1r2wlnUQ#Actions-decided-on-20190522). d. Maria asked for a new _stagiaire_, hopefully coming mid-September, to apply, test, identify shortcomings of the [Instructions](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#) and turn them into a Markdown web site - suggested name: https://documentation-instructions.docs.cern.ch. Please see [internship project proposal here](https://it-student-projects.web.cern.ch/projects/documentation-project-migration-static-web-sites-markdown). e. Nikos will give the White Area lecture on our Documentation Project to CDA on **Monday September 30th**. Maria prepared [event](https://indico.cern.ch/event/829649/). Please enhance the description. All documentation-project e-group members have management rights in this event. f. Alex's summary of the current limitations in a. and b. above: The command used was (for Nikos' [Instructions]((https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#) - the web UI could be used as well): ``` oc create route edge mailservices.docs.cern.ch --insecure-policy=Redirect --hostname=mailservices.docs.cern.ch --service=mailservices-docs ``` For reference, there are still some limitations. This is just a workaround and full support for docs.cern.ch will need a complete rewrite of webservices (or a manual setup). We can now use docs.cern.ch in the following conditions: 1. Openshift "official" site using the Instructions](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#). 2. SSO requires that the same base name is used in the docs URL (i.e if the site is created initially as sitename.web.cern.ch, it can use SSO for sitename.docs.cern.ch but not for othername.docs.cern.ch) fixed-telephony-service.web.cern.ch and it-e-learning.web.cern.ch are hosted in webeos and do not meet the first criteria. mailservices.docs.cern.ch cannot use SSO yet because it does not meet the second criteria (SSO would work with mailservices-docs.docs.cern.ch though, since the initial URL is mailservices-docs.web.cern.ch). g. Very positive comments from IT [see here](https://mattermost.web.cern.ch/it-dep/pl/j7gmmojqujrcbr4rq5aknm67hh). ## New timelines 2019/06/03 Following a discussion between the Documentation Project managers (Nikos K. & Maria D.) the individual emails to the ASDF members from IT will be sent **during the week of June 24th** by Nikos (currently on holiday). Reasons: a. Alex's implementation of <service>.docs.cern.ch, including SSO access for restricted/admin pages will be complete. b. Nikos should test (with SSO included) before changing the [Instructions](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#). c. Maria currently editing OpenShift-defined site https://mailservices-docs.web.cern.ch/ met performance problems to pull/push on 29/5 and this morning 3/6. It would be good to continue this pilot usage _until_ we reach smooth operation and _before_ inviting others (outside CDA). Nikos will **not** give the June 24th White Area lecture to CDA (his first working day). Info sent by Maria to Tim, Emmanuel, Andreas, Thomas, cc: Nikos. ## News on 2019/05/23 Following Emmanuel's feedback to Maria, the text for the email to ASDF members from IT is changed and will be sent once: a. Alex finishes the implementation (including SSO access) b. Nikos updates the [Instructions](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#) to reflect the **.doc** site definition and the SSO possibilities. ## Actions decided on 2019/05/22 Taken with f2f discussions (no meeting): 1. Nikos to _test_ the method, implemented by Alex, to configure a web site in OpenShift so that it appears as <service>**.docs**.cern.ch and _update_ the [Instructions](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#) to include this possibility, even though SSO is not yet implemented. Niko, please substitute 'project' with 'service' to use the same terms everywhere. 2. Maria to email the appended text to ASDF members _from IT Dept only_ willing to test and send us feedback. To discuss the feedback we'll receive, we shall organise our next checkpoint meeting. Documentation-project members please **comment/correct** this message text: ``` Dear IT service managers, thanks to the good work of our Paas, gitlab and web authoring developers, we are now able to invite you to follow these instructions https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ# and create a static Markdown web site in OpenShift, called <service>.docs.cern.ch. SSO access is also possible. Please test and send us your feedback. Thank you very much for your interest in the CDA Documentation Project. ``` ## Status on 2019/05/08 Applied Nikos's [Instructions](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#) for OpenShift-based site creation with the Mkdocs boilerplate and the gitlab configuration for builds running on OpenShift to create site https://cern.ch/mailservices-docs. Thanks are due to Nikos for clear instructions and help with novice user (Maria). To discuss: Will existing sites now named <service>-docs.cern.ch be automatically migrated to <service>.docs.cern.ch? No! Hence, we should go to the <service>.docs.cern.ch directly. ## Actions decided on 2019/05/08 1. Nikos to update the [Instructions](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#) to include the <service>.docs.cern.ch, when Alex has tested and put in operation the SSO access possibility. 2. It should be clear in the instructions that the site should be defined in webservices as <service>-docs.cern.ch but when the owner changes the route in OpenShift, it will be changed to <service>.docs... This will also appear in the Instructions when point 1, above, in done. 4. Re-discuss the restricted parts of a given documentation site later. At this moment the recommendation, for implementation purposes becomes "create a new site for Administration" pages. 5. New documentation sites from within our group should be migrated to <service>.docs.cern.ch. 6. Walk through Nikos's [Instructions](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#) at the [ASDF a.s.a.p.](https://indico.cern.ch/category/7664/). This will give us IT input to fix things and open-up to others, e.g.ATLAS, who are already interested. 7. About _Editors_ the draft shortlist by Nikos is [here](https://codimd.web.cern.ch/s/HJR1MdyY4#). A few testers will be approached to evaluate an give us feedback. 8. About _MkDocs themes_ we now have only the _Material_ theme in the template, which is the default, and another two (MkDocs and ReadTheDocs). Michal said some more themes are desirable. He will send the ones he wants to Nikos. 9. After point 7 above, Nikos will update the Instructions to guide newcomers for a theme selection. 10. How about organising [a White Area](https://indico.cern.ch/category/7664/) to inform our CDA colleagues and inspire them to embark in this effort? Next meeting: After ASDF presentation and subsequent feedback. Possible dates for the presentation 23 May or 20 June with Maria and Emmanuel (Nikos being away) **IF** the SSO implementation for <service>.docs.cern.ch is tested and in operation. ## Policy on FAQs 2019/05/02 Following two White Area discussions [on 2019/02/25](https://indico.cern.ch/event/801306/) and [on 2019/04/29](https://indico.cern.ch/event/801864/) about future FAQ locations the policy conclusion agreed with the CDA management on is: 1. We need to find out whether there is a API to synchronise between SNow KBs and Markdown service site sections with FAQs. Thus, Markdown sites can contain everything without the service managers having to maintain stuff in many places. Ideally with a flag signaling what has precedence (SNow or the Markdown site). 2. Discourse remains the recommended tool for community peer help. ## Status on 2019/04/15 * The user acceptance evaluation decided at the late meeting was converted to editor evaluation and will be done in the near future, when we know which shortlist of editors we wish to try. * Discussion on editors' recommendations was left for the next meeting. Alex mentioned https://www.netlifycms.org/ * On how to create a new documentation site in OpenShift. Nikos's [steps presented here](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#). * It is now impossible to hung the new site name under the .docs.cern.ch domain. We can't use the existing machinery of creating a site under .web.cern.ch. It has to still be done manually. To accelerate the process of this design and development Bruno suggests a sprint on how to add <myservice>.docs.cern.ch sites automatically in DNS. **Should we do this?** * The **restricted access** to some sub-directories (=specific documentation sections) was discussed. Alex & Bruno find the implementation complicated in OpenShift and suggest independent sites for public and admin pages. Others explain that this is what we recommended, already implemented in some cases, e.g. https://cern.ch/it-e-learning/administration and some services will eventually require it, e.g. SNow Level 1,2,3 documentation. * Scripts for automatically mapping gitbook syntax to mkdocs was left as an exercise to expert IT service managers with existing gitbooks willing to move. ## Actions decided on 2019/04/15 1. Nikos - finalise the steps for Openshift-based site creation (currently in openshift-dev). [DONE here!](https://codimd.web.cern.ch/0kWaj917TT6jJJzu3lPZCQ#) 2. All - confirm/approve steps above. 3. Nikos & Alex - Publish MkDocs S2I on OpenShift production. Next meeting: **Thursday May 9th at 9am NB!** unusual time! ## Status on 2019/04/01 Progress in the last 2 weeks: * docs.cern.ch created. Need to issue the certificate. * Nikos & Alex discussed the OpenShift web site creation & the gitlab repo. They recommend the CI creation in OpenShift (with S2I). Important reason for this choice is the installation of security updates and maintenance of the CI, which wouldn't happen in the case of gitlab CI, unless the user/owner pushes an update. * Automation of the gitlab & OpenShift steps to hide the complexity is possible but when the user only sees _StackEdit_ (or other editor) and anything breaks in the series of automated steps the debug will come to us. * Tim suggests that we envisage Markdown-based documentation for all the thousands of services defined in the SNow service catalogue, hence we need to know the feasibility of automation of the whole process. * Physicists use _Overleaf_ and they never actually see gitlab. How can we do this 'masking' for general service documentation? ## Actions decided on 2019/04/01 1. To evaluate user acceptance Nikos and Emmanuel will start by getting feedback from a few non-IT users (e.g. the cern library) on the use of Markdown (with a selection of editors), to start with. 2. Move Maria's [it-e-learning](https://cern.ch/it-e-learning) site to test the 5 steps of automation and change its gitlab location and the web server from EOS to OpenShift. * **Next meeting: April 15 @ 2pm in the Social Room** ## Status on 2019/03/18 Implementation discussion on the [ASDF outcome](https://indico.cern.ch/event/800408/), mainly the gitlab environment for <service>.docs.cern.ch. The debate was what/if/how to automate from the following: 1. infrastructure for web site creation <service>.docs.cern.ch 1. OpenShift (we used EOS so far for some cases) 1. gitlab repo + CI -> Mkdocs template ![](https://codimd.web.cern.ch/uploads/upload_962c96d4b24f9a2f273222e106881934.jpg) ## Actions decided on 2019/03/18 1. Alex & students will offer a new button _documentation site_ from [the OpenShift console](https://openshift.cern.ch/console/catalog). 2. Alex will check what the gitlab plans for a functional Markdown editor are and inform us. 3. The webservices developers will make available a new documentation-specific option on [the create new site page](https://webservices.web.cern.ch/webservices/Services/CreateNewSite/Default.aspx) 4. Nikos will follow [this recipe](https://discourse.web.cern.ch/t/deploy-mkdocs-documentation-on-openshift/565/2) to create the template. 5. Project management by Nikos+Maria 6. Consider [suggestions by Tim Bell in the ASDF mattermost channel](https://mattermost.web.cern.ch/it-dep/pl/ikpbfgc5z7fg58rqyigg4kj1ir) namely: a. add an automatic 'help us improve the docs' block ? This is nice to allow users of documentation to submit improved content. b. automatically add an appropriate license statement also in the template? (see discussion on ~FOSS at CERN ) Next meeting: **Monday April 1st at 2pm in the Social room** . Created [e-group documentation-project](https://e-groups.cern.ch/e-groups/Egroup.do?egroupId=10325469). ## Status on 2019/03/14 [ASDF Presentations on Documentation](https://indico.cern.ch/event/800408/) inventory of tools (by N.Kasioumis) and Service Documentation (by M.Dimou). Participants ask for: * search across documentation * gitlab infrastructure * gitbook to mkdocs migration * SNow KB content * a technical writer who produces documentation There was a consensus that CDA gives priority to the creation of the gitlab and OpenShift environments for service managers IT-wide to start moving their documentation there. ## Status on 2019/02/20 [Slides](https://codimd.web.cern.ch/p/HkV-909H4#/) FAQ-specific for the A.O.B. of the White Area lectures kick-off Mon 25/2 @ 3pm. ## Status on 2019/02/18 Discussion in view of gitbook disappearence from the public domain (Tim, Thomas, Sebastian, Emmanuel, Nikos, Maria D.) for a future ASDF **14 March @ 4pm** . * Internal note: We shall probably recommend an OpenShift site, not EOS. ==> No need to create a service account. **Question:** Please do inform Maria, if this comes true, as this means that the [Service Documentation Guidelines](https://twiki.cern.ch/Edutech/ServiceDocumentationGuidelines) must be changed. * **Action:** Set some dates for the one-click preparation of the environment (default domain name, gitlab, web server). * We used no tool to convert the Indico doc from gitbook to MkDocs. The main difference is the way MkDocs uses to build the table of contents [see here an example](https://gitlab.cern.ch/dimou/it-e-learning/blob/master/mkdocs.yml) and [its published result](http://cern.ch/it-e-learning). * Sharepoint will die (in <=3years). * FAQs should not be mentioned in ASDF. **Action:** FAQs should be further debated internally in CDA. * **Action:** Maria & Nikos - 10' mins each to leave room for discussion. Content: Nikos sets the landscape with popular tools in use today. Maria lists what we use and agreed to use in the CDA Documentation strategy (Markdown with MkDocs and gitlab). Twiki running service, sharepoint faceout, web dfs/afs out, eos/openshift ok. SNow is ok for KBs. Organised doc. is more complete and structure. * No policy statement today about the value of _service_ in the future _service_.doc.cern.ch. ## Discussion on FAQs 2019/02/07 Discussed the CDA Doc. strategy with N.Kasioumis & N.Kane. Natalie rightly reminded: * On FAQs: * The SNow KBs send reminders prompting for update. The user-facing documentation in Markdown won't. * The SNow KBs can be updated by the supporters, not only the service managers. We'd have to persuade the supporters and complicate edit rights in gitlab so they can update the FAQs in Markdown. * SNow sends automatically notifications about new KBs showing up. The user-facing documentation in Markdown won't. * On replacing currently existing sites with Markdown: * printing, account, cerntificate, mail & other group service pages today run scripts that show info relevant to the user's login or IP. * because of the above sinequanon for adoption of the strategy some funds should be found for hiring the student [job description](https://it-student-projects.web.cern.ch/projects/malt-related-project-standard-documentation-workflow-and-conversion-tools-documentation) who was refused during the last Technical students' selection in the autumn 2018. * alternatively, we could make a few Working Groups doing _sprints_ for the porting of dynamic features from today's service web sites into thew new documentation. **2019/02/07**: Question submitted to the SLM today: * On 2018/11/21 we had 3 votes in favour of the creation of the format <service>.docs.cern.ch URLs (Maria, Tim, Thomas) to facilitate Public Service Documentation identification by the user community. * when selecting the <service> name Natalie suggested we should take into consideration the [SNow Customer Service names in the Service Catalogue](https://cern.service-now.com/service-portal/service-area.do?name=IT-Services). Should we? * Emmanuel had said that the gitlab environment would be prepared (replication of the boilerplate, web server, service account etc). * The above 2 bullets are not done (I discussed with Andreas and Bruno). Please give management directive for one of Bruno's students to work on this. I have made the cookbook but the process is not easy for newcomers to git. **2019/02/07**: Reviewed _FAQs location_ strategy: Not _discourse_, into the _Public Service documentation_. Approved by the [CDA SLM meeting on 2019/01/24](https://indico.cern.ch/event/787521/). [Slides updated](https://codimd.web.cern.ch/p/Hy3SjYE2X#/12). Full rationale [here](https://indico.cern.ch/event/791304/contributions/3286501/note/). **2018/11/07**: Discussion at the CDA/IC section meeting [General section information](https://indico.cern.ch/event/770306). **2018/10/31**: Summary [slides endorsed by a dedicated CDA SLM](https://hackmd.web.cern.ch/p/Hy3SjYE2X#/) . **2018/09/20**: [Technical Student](https://it-student-projects.web.cern.ch/projects/malt-related-project-standard-documentation-workflow-and-conversion-tools-documentation) (refused) and [Stagiaire Catarina Vieira](https://it-student-projects.web.cern.ch/projects/malt-related-project-new-documentation-testing-dateness-and-functionality) (found and coming on 2019-01-07 for 2 months at 50%) project proposals to help with the implementation. **2018-08-23**: Initial draft document discussed between Tim, Emmanuel, Thomas & Maria D. in the appended text. This page briefly lists the types of documentation every Service needs to have and where each of them should live. ## Purpose These guidelines are established as a reference for service managers in order to ensure: - a common look and feel for all user-facing documentation of CDA services - that all service documentation is independent from a temporary administrative structure - that central tools are used whenever possible - that publishing and maintaining this information is as easy and standardised as possible - as little data redundancy as possible ## Proposed Guidelines Each CDA service should implement the following documentation 1. **Short General Service Description**: SNow FE/SE Description and the relevant bubble of https://cern.ch/it-dep-cda/services/ (currently manually synchronised, hopefully automated in the future). short service description on SNow: [![short service description on SNow](https://hackmd.web.cern.ch/uploads/upload_da9cb7750ad8ab201ea805610ca7f6e5.png =350x)](https://cern.service-now.com/service-portal/service-element.do?name=Indico-Service) short service description on the CDA site [![Short Service Description on the CDA site](https://hackmd.web.cern.ch/uploads/upload_815a513119cf7e76cd8f94941ae94c69.png =350x)](https://it-dep-cda.web.cern.ch/it-dep-cda/services/indico/) 1. **Detailed Service Description**: Dedicated Service Site which URL should not contain any reference to the organisational unit currently running the service, and which content should include: - Public service web site and/or User Guide (Markdown) _for all users_ - public or restricted to CERN users, example [the Indico User Guide](https://indico-user-docs.web.cern.ch/indico-user-docs/) [![Service User Guide](https://hackmd.web.cern.ch/uploads/upload_98b51de2093b9ab93ab16b7334b45417.png =350x)](https://indico-user-docs.web.cern.ch/indico-user-docs/) - Administration Guide (Markdown) _for service managers' internal use. It can be hosted on the same web site but shouldn't be linked from the public view._ - restricted to service administrators, example [the Indico Ops Guide](https://indico-ops.web.cern.ch/indico-ops/). [![Service Administration Guide](https://hackmd.web.cern.ch/uploads/upload_1d060e9bd81a488117bad95e16cd48f1.png =350x)](https://indico-ops.web.cern.ch/indico-ops/) 1. **FAQs**: SNow KBs _for all users and Level 1,2 supporters_ [![Service FAQ](https://hackmd.web.cern.ch/uploads/upload_c9898728e3f144f9d7915b87c0b51a63.png =350x)](https://cern.service-now.com/service-portal/faq.do?se=Indico-Service) 1. **Service Incidents/Changes**: SNow SSB [![](https://hackmd.web.cern.ch/uploads/upload_9b89e5e67c77f9ed58ce2b558efd4840.png =350x)](https://cern.service-now.com/service-portal/ssb.do?area=IT) 1. **Change Requests**: Tracked in either Github/Gitlab Issues or Jira tickets, and entered either directly or through SNow requests. [![](https://hackmd.web.cern.ch/uploads/upload_2bdbef664cc0e34d7fbe228cded925f0.png =350x)](https://github.com/indico/indico/issues) 1. **Privacy Notice**: SNow Please check [this cookbook](https://twiki.cern.ch/Edutech/ServiceDocumentationGuidelines) that contains the basic steps for doing point 2 above. ## How these guidelines were born - [IC section internal reflection](https://hackmd.web.cern.ch/s/rJXzcvzem#) - [Group-wide reflection](https://twiki.cern.ch/CDAgroup/CDAPublicDoc) - [Technical notes](https://twiki.cern.ch/Edutech/ServiceDocumentationGuidelines) on Markdown editing, gitlab publishing and EOS hosting