Notes for Franciska's BSc thesis === **Random notes, as they occur, not to forget, at the time of thesis report writing** ## Project Description: https://it-student-projects.web.cern.ch/projects/cern-academic-training-web-site Maria's text on separation of time spent as _Internship_ and as _BSc thesis work_, to which the VIA University thesis supervisor Kasper Rasmussen agreed: a. Internship work - August 2021 - January 2022 - 6 months: The learning process of Linux system administration, React, Django, Kubernetes, OpenShift, the CERN Web Frameworks. Franciska has a CERN Udemy license for any technical and/or managerial course, potentially useful for her future career. She also follows the CERN french language courses. I introduced Franciska to the arts at cern officer and approved an Adobe packages' license for her web development project and for other photos' editing for publication to the CERN Document Server (CDS). b. BSc thesis work - February -July 2022 - 6 months: Application of the acquired knowledge with the above-mentioned tools to actually implement the web site according to the project description above. Scripting for video fetching from CDS, development of the Search function, performance testing, entry in operation, documentation, enhancements based on users' feedback. ## The repository for the project: https://github.com/cern-sis/cern-academic-training ## The development tracking system https://webservices.web.cern.ch/webservices/Services/SiteDetails/?SiteName=academictraining Use these items in the BSc thesis report. ## The mockups' public location https://cernbox.cern.ch/index.php/s/HNlL4UqMXYPheal Place the mockups in the thesis as local images. The directory is public but might be erased when your CERN login will expire. **Mockup prototype:** Desktop: https://xd.adobe.com/view/69a67c1c-f030-4c84-82d1-fe8a5ea66987-3cf5 Mobile: https://xd.adobe.com/view/643fc215-f4c5-4a79-a654-67a7283e03e1-316c/ ## Photos Place [this link](https://cds.cern.ch/search?ln=en&p=toeroek&action_search=Search&op1=a&m1=a&p1=&f1=&c=CERN+Document+Server&sf=&so=d&rm=&rg=10&sc=1&of=hb) in the thesis. This collection is public, so you don't have to copy anything... I don't think it is related to your CERN login expiration. ## Our Group and Section Group https://cern.ch/it-dep-cda Section https://cern.ch/it-dep-cda/group/ic/ - The person in the middle is our leader _Thomas Baron_. Services https://cern.ch/it-dep-cda/services/ Use mattermost for communication with: Web Frameworks experts: Michal Kolodziejski, Andreas Wagner, Aleksandra Wardzinska (our floor) Web search experts: Ismael Posada, Pablo Panero (our floor) Drupal experts: Ismael Posada, Konstantinos Samaras-Tsakiris (our floor) React experts: Rene Fernandez Sanchez, Michal Kolodziejski Web lectures' experts: Rene Fernandez Sanchez, Ruben Domingo Gaspar Aparicio (upstairs) CDS experts: Nicola Tarocco, Karolina Przerwa (our floor) ## Learning courses: 1. French Integration course for beginners: **[LINK](https://lms.cern.ch/ekp/servlet/ekp?PX=N&TEACHREVIEW=N&PTX=&CID=EKP000043063&TX=FORMAT1&LANGUAGE_TAG=fr&DECORATEPAGE=N)** 2. Linux Mastery: Master the Linux Command Line in 11.5 Hours: **[LINK](https://cern.udemy.com/course/linux-mastery)** 3. Learning Linux Essentials: Taking your first steps in Linux: **[LINK](https://cern.udemy.com/course/learning-linux-essentials-taking-your-first-steps-in-linux)** 4. Python Django 2021 - Complete Course: **[LINK](https://cern.udemy.com/course/python-django-2021-complete-course)** 5. React and Typescript: **[LINK](https://cern.udemy.com/course/react-and-typescript-build-a-portfolio-project)** 6. Kubernetes: **[LINK](https://cern.udemy.com/course/learn-kubernetes)** 7. Openshift: **[LINK](https://cern.udemy.com/course/learn-openshift)** 8. New Udemy signed on 2022/03/29. List the additional courses taken until the end of contract. ## Links for the thesis report 1. The project URI, of course 2. [An Academic Training series on Kubernetes](https://indico.cern.ch/event/829468/). It is 2 years' old. Use only if it has useful info. 3. [Aristofanis's thesis](https://cds.cern.ch/record/2724152?ln=en) _maybe_, if the info on React can be used, with no plagiarism. 4. The CDS photos used for the site. 5. https://indico.cern.ch/event/1106815/ - it has nothing to do with the project but it contains material super interesting for the change of our CERN IT env. from research to business. Very useful for the report! ## Logo ideas [Director's cut](https://www.onlinewebfonts.com/icon/524690) and an [academic cap on top of it](https://www.comarfi.com/en/graduation-and-end-of-course/academic-cap-1662.html). ## SNOW tickets 1. [On Number of Views](https://cern.service-now.com/service-portal?id=ticket&table=u_request_fulfillment&n=RQF1900855) 2. [On fetching the views](https://cern.service-now.com/service-portal?id=ticket&is_new_order=true&table=u_request_fulfillment&sys_id=05e0a3861b97b0107a83dc6a9b4bcbb1) 3. [On JIRA project _Academic Training web site permissions_](https://cern.service-now.com/service-portal?id=ticket&table=incident&n=INC2957974) 4. https://cern.service-now.com/service-portal?id=ticket&table=u_request_fulfillment&n=RQF1900855 ## Tracking with JIRA It will be useful also for the thesis report to have the JIRA tickets to refer to. Definitions of JIRA terms epic VS story VS task: Epic - something so big it probably won’t fit into a sprint, is not clearly understood in terms of customer requirements and should be broken down into stories. T-shirt sizing is a common way to size epics. Another way is to say we think it could take X to Y iterations to do this work. Epics are usually defined during initial product roadmapping and decomposed into stories in the product backlog as more is learned. Story - something actionable and small enough to fit in a sprint. These are story pointed and defined using INVEST criteria. Stories should deliver a vertical slice of functionality to the customer that is valuable and complete by the end of an iteration. Stories are usually created throughout product development, more so leading up to iteration planning and also during higher level product roadmapping. Tasks - decomposed parts of a story that get into HOW the story will be completed. Tasks can be hour estimated if desired. Tasks are usually defined by the people doing the work (developers, QA, etc), whereas stories and epics are generally created by the customer or the product owner representing the customer. Tasks are created within the bounds of an iteration because they are short-lived. There is very little value is tasking out stories that won’t be worked on in the upcoming iteration. Ideally we should have sprints with deadlines and epics, stories, tasks have size and duration. ## 2021-08-05 * CERN guidelines' compliance for making a CERN web site is a _must_. * License issues with usage of tools and material. OK purchased. * Can Franciska, being a photographer, use her own photos or should she take the ones in CDS https://cds.cern.ch/. Yes. * Search algorithm to include keywords from the _transcript_ of the AT videos. This won't be available for several months. **Pending** * Does the site have to be in french as well? No. * CERN Document Server - _Academic Training Lectures_ - use the API of CDS to get all the academic lectures. * Decide about technologies - most of CERN sites use Drupal and Bootstrap - these are the _must_? - because CERN Theme is based on Drupal * Provide short video previews on the site 2''-3'' secs. * webcast.cern.ch check how it looks... * https://design-guidelines.web.cern.ch/ * Am I allowed to make a new logo or only text for the Academic Training Lectures? Yes. * Color Palette options: https://indico.cern.ch/event/1066249/ (material) ## 2021-08-17 * Mention in the thesis, the ATC meeting of 2021-08-17. Access to https://indico.cern.ch/event/1066249/ is restricted. Still there can be a narrative explaination on "how you obtained consensus with the content owners". * Notes from the meeting: https://indico.cern.ch/event/1066249/note/ ## 2021-08-11 Meeting with Web Frameworks' experts Andreas Wagner and Michał Kołodziejski <michal.kolodziejski@cern.ch>: * On the _CERN Guidelines_: Stick to them given that the site is official. * On search: Collaborate with Ismael Posada and Pablo Panero. * On photos: Go ahead and make them, the site is copyright cern anyway, so Franciska _offers_ her photos to CERN. Next check with the ATC. * rething the category button visibility (check with Michal next week if needed). * The _dots_ for the carrousel: Replace by arrows left/right. Much more functional for the mobile version. * **Drupal** experts: Ismael Posada and Konstantinos Samaras-Tsakiris. * **React** is the technology recommended for _front-ends_ of new web sites. It leaves all the _back-end_ to the developer. Check React doc on the web to make an opinion. * For the back-end: The Notifications project uses Node JS, Indico uses python and JS. * Drupal has _search_, _keyclock login_, _comments_ and more ready out of the box. * **Wordpress** is also available but is very limited in functionality and doesn't address the back-end problem. * **ACTIONS Franciska** 1. Read [the thesis of the CERN Slides' app](https://cds.cern.ch/record/2724152?ln=en). The front-end is React. Check the back-end technology. 2. Make a comparative table Drupal vs React depending on the required web site features for her design. ## 2021-08-17 **ATC meeting** - see preoccupations in [the notes](https://indico.cern.ch/event/1066249/note/). New mockups, reflecting this feedback, will be uploaded a.s.a.p. on the same page. **CDS experts** - Discussed with Nicola and Karolina. Their recommendations: 1. Make a database (db) to hold the video metadata, local to the site. This will improve performance. 2. The video _embed_ will continue working from CDS. 3. Alternatively get the video directly from the (now DFS), future CephFS location. * Franciska, to check [this White Area presentation material](https://indico.cern.ch/event/1041657/) to understand the new architecture. 5. Sync nightly with this local db for newly published videos. 6. Once the web site technology is decided (Drupal?) run performance tests with a few videos. 7. Check how videos with camera+slides on different channels behave. 8. Use [the _paella_ player](https://paellaplayer.upv.es/). This is our preferred player. Expert is René (see experts' list above). 9. Franciska regularly check with Nicola for advice concerning the CDS interfacing. ## 2021-08-18 Technology discussions **Action for Maria** [The Research Sector website](https://cern.ch/rcs) is made with Drupal. For future maintenance, we should involve the webmaster earli..ish. Maria to discuss with the Directorate and Ismael, [who are the drupal-rcs-admins](https://webservices.web.cern.ch/webservices/Services/SiteDetails/?SiteName=rcs) (e-group is private). **The React repo** of the [webcast](https://webcast.web.cern.ch) site is https://gitlab.cern.ch/webcast/webcast-website/-/tree/master/ **Action for Franciska** Make a table of comparison between the various technical solutions. Add _Invenio_ in the list. **With Nicola Tarocco at least 3 options have been discussed for the technology:** * Drupal * hybrid: Invenio -> https://inveniosoftware.org * full custom: React + Python (Django) ## Technology alternatives' table | Frontend | Back-end | Advantages | Disadvantages | Comments | | ---------- | --------------- | ----------- |--------------- | -------------- | | Drupal | | Easier - maintenance, deployment, authentication, data storage provided by CERN | Not going to learn much, since most things are implemented + not much to write in BCs thesis + detrimental for long-term | | | React | Django (Python) | Will learn a *lot* and BCs thesis can be written properly + advantegous for long-term | After finishing must be maintained by someone + more complicated (has to be built from scratch) | Check [this BSc thesis](https://cds.cern.ch/record/2724152?ln=en), also on your desk in paper. It was done in React and Openshift 3. Now all is OKD4, meaning some of the info may be out of date. | | Wordpress | | | | Depending on the VIA univ. BSc content requirements, you may need to evaluate this option for the point in [the project](https://it-student-projects.web.cern.ch/projects/cern-academic-training-web-site) saying:_Using the same Markdown-based documentation policy make a technical recommendations' document for the end-users https://makeyourownsite.docs.cern.ch, with instructions, based on the technologies available from IT/CDA-WF._ | | Invenio | | | it is in early development and has the same situation as Drupal | Same comment as above. | ## 2021-09-22 About Us page [ATC Members](https://hr.web.cern.ch/academic-training-committee-atc) * Make a list of members with dynamic profiles ## 2021-09-23 Logo options 1. Academic cap on top of CERN logo  2. Director's cut with academic cap on top of it. ## 2021-10-11 Meeting with **Harris Tzovanakis and Salome Rohr**, RCS webmasters 1. The CERN Research and Computing Sector (RCS) have one site in Drupal. 2. React for the front-end and Invenio for the back-end is used for other sites. 3. They have their own Kubernetes cluster and advise to build the house there, if it is for maintenance by them indeed. 4. A back-end is needed for the site and the use of a real database to retrieve CDS data and store locally for performance. )Django is suggested by Harris. 5. Maria to check with their supervisor on point 3. The proposal for webmastering in RCS has the Research Deputy Director's agreement. 6. Maria checked with CDS experts the availability of OAI-PMH. The answer is **Yes**. A meeting with Kasper Knop Rasmussen (KASR) VIA University <kasr@via.dk> showed that the React approach meats positive reactions from all sides. ## Site definition Harri, to match the architecture you are familiar with, where should the site be defined? On https://paas.cern.ch for OKD4 base? We, Academic Training Committee (ATC) would like site name https://cern.ch/academictraining. ## Site examples Can Franciska clone one of the appended CERN gitlab repos and use as example for her implementation? * https://catalogue.library.cern/ created by the IT CDS team. Frontend is React. Backend is Invenio. * https://videos.cern.ch created by the IT CDS team. Front-end is javascript, which should be enough for us. Also some AngularJS there (old technology, deprecated), for which we don't care but put here for completeness. Backend is Django. Check [this info by Nicola Tarocco](https://mattermost.web.cern.ch/it-dep/pl/7uatg94qetrgjffmotbk8uqdph) on CDS data extraction. * https://inspirehep.net/ is created by Harris and the RCS team. See a presentation about it in https://indico.cern.ch/event/896545/  ## About search From [the slides of Antunes and Lourenco linked from this page](https://indico.cern.ch/event/1006978/timetable/#20211012.detailed) some hints on the web site's _search_ function:    The search of the very few CDS video records for Academic Training (hardly 1K) don't require a complex search algorithm, as Indico does (>> x100M searchable items, events or materials). Still, it would be nice for Franciska to see the packages used and the experts involved. ## About CDS extraction From Nicola Tarocco in mattermost: You can use OAI-PMH to fetch the lectures if it easy for you we have already an export set up for the collections Academic Training Lectures and Summer Student Lectures you can use that one, or we can create a specific set if you need other collections (a set is simply a specific search query that allows you to export records based on field/tag) Here the links (with the different export formats): DublinCore: https://cds.cern.ch/oai2d?verb=ListRecords&metadataPrefix=oai_dc&set=forSciTalks&from=2021-10-01 MarcXML: https://cds.cern.ch/oai2d?verb=ListRecords&metadataPrefix=marcxml&set=forSciTalks&from=2021-10-01 In your browser, you have a “nice” UI that can help you, if needed: http://cds.cern.ch/oai2d?verb=ListRecords&metadataPrefix=marcxml&set=forSciTalks&from=2021-10-01 Do not request ALL records because it is very slow, you should use pagination, I will correct the URLs above in a minute. Query updated with the from parameter. ## 2021-10-13 Meeting with Harris We have discussed with Harris the following: - MarcXML metadata from CDS will be converted to JSON for simplifying data, for example: https://codimd.web.cern.ch/gYTkgkaBQQa4iuLnAa3RwA - There is going to be a REST API established with only the necessary fields (like title, author, abstract, etc) from CDS - a dataset with necessary schemas - Back-end technology is going to be Django (Python) and the Front-end technology React App. - Several steps have been elaborated - the first one is to select the most necessary fields from CDS which we need for the website (both for displaying and for storing) - the rest of the steps will be discussed later - CERN Authentication is needed - Meetings/checkpoints everyday at 11AM with Harris ## 2021-10-14 ATC Meeting: - instead of "Technology" category put "Applied Physics" - Full notes https://indico.cern.ch/event/1068065/note/ only viewable from atc-members e-group. Franciska is included for this development. Meeting - Harris: - create database with Django **DONE** - MARCXML mapping for the database: example: https://cds.cern.ch/record/2774582/export/xm?ln=en ``` ID <controlfield tag="001">2774582</controlfield> TITLE <datafield tag="245" ind1=" " ind2=" "> <subfield code="a"> DATE <datafield tag="269" ind1=" " ind2=" "> <subfield code="c">2021-07-01</subfield> CORPORATE AUTHOR(S) <datafield tag="110" ind1=" " ind2=" "> <subfield code="a">CERN. Geneva</subfield> </datafield> ABSTRACT <datafield tag="520" ind1=" " ind2=" "> <subfield code="a"><!--HTML--> blablabla </subfield> SERIES <datafield tag="490" ind1=" " ind2=" "> <subfield code="a">Academic Training Lecture Regular Programme</subfield> <subfield code="v">2020-2021</subfield> </datafield> SPEAKER <datafield tag="700" ind1=" " ind2=" "> <subfield code="a">Tovey, Daniel</subfield> SPEAKER DETAILS <datafield tag="700" ind1=" " ind2=" "> <subfield code="u">University of Sheffield</subfield> EVENT DETAILS <datafield tag="856" ind1="4" ind2=" "> <subfield code="u">https://indico.cern.ch/event/870610/</subfield> THUMBNAIL PICTURE <datafield tag="856" ind1="7" ind2=" "> <subfield code="2">MediaArchive</subfield> LANGUAGE (Maria said that there are proposals to accept famous speakers only willing to speak french. So it is recommended to have it included.) <datafield tag="041" ind1=" " ind2=" "> <subfield code="a">eng</subfield> </datafield> SUBJECT CATEGORY <datafield tag="650" ind1="1" ind2="7"> <subfield code="a">Academic Training Lecture Regular Programme</subfield> </datafield> LECTURE NOTE <datafield tag="518" ind1=" " ind2=" "> <subfield code="d">2021-07-01T11:00:00</subfield> </datafield> IMPRINT (DURATION) <datafield tag="269" ind1=" " ind2=" "> <subfield code="c">2021-07-01</subfield> </datafield> <datafield tag="300" ind1=" " ind2=" "> <subfield code="a">0:56:59</subfield> </datafield> COPYRIGHT/LICENSE (I'm not sure about this one, it was hard to find it - please verify it) <datafield tag="542" ind1=" " ind2=" "> <subfield code="d">CERN</subfield> <subfield code="g">2021</subfield> </datafield> ``` ## 2021-10-19 **SQL Query suggestion for Database:** ``` CREATE SCHEMA academictraining; SET SCHEMA 'academictraining'; CREATE TABLE lecture ( lectureID SERIAL PRIMARY KEY, title VARCHAR(100) NOT NULL, date DATE NOT NULL, corporateAuthor VARCHAR(100) NOT NULL, abstract VARCHAR(3000) NOT NULL, series VARCHAR(50) NOT NULL, speaker VARCHAR(50) NOT NULL, speakerDetails VARCHAR(50) NOT NULL, eventDetails VARCHAR(50) NOT NULL, thumbnailPicture VARBINARY(MAX) NOT NULL, language VARCHAR(3) NOT NULL, subjectCategory VARCHAR(50) NOT NULL, lectureNote TIMESTAMP NOT NULL, imprint TIME NOT NULL, license VARCHAR(30) NOT NULL ); ``` **Meeting - Harris - in presence of Maria:** - be careful, the oldest ones do not have videos! - some old records have only PDFs - no categories - since categories have not been identified earlier in CDS, the only possible solution for that it would be Machine Learning to declare the most common keywords - no Machine Learning for now - it is not declared in the Project Description as a requirement - views cannot be fetched from CDS metadata - according to Harris: harvest only once in the beginning (it takes days) - do not harvest several times - no synchronization between CDS and AT website - Harris' suggestion: new website with only the videos (because there are PDFs as well) - everyday fetch the new videos - but no updates from older lectures - if PDF could be still displayed on the website, a particular slider/viewer would be used for that - for the views we can abandon the CDS records, and start from scratch on the website example: (70' old record): https://cds.cern.ch/record/1197992/export/xm?ln=en example: (new record) https://cds.cern.ch/record/2782916/export/xm?ln=en ## 2021-10-21 - Clone repo (git) - **DONE** - Solve _yarn install_ - **DONE** - Solve python version (in order to be compatible with _poetry_) - **DONE** - Open [SNOW Ticket](https://cern.service-now.com/service-portal?id=ticket&is_new_order=true&table=u_request_fulfillment&sys_id=05e0a3861b97b0107a83dc6a9b4bcbb1) for returning video records in OAIPMH API - **Pending** - Create new JIRA project for Academic Training website - Franciska doesn't have permission to create a new one. **Done by Maria.** ## 2021-10-22 Now that https://its.cern.ch/jira/projects/ATWEB is created, Franciska could take the items from the above discussions that are development items for the project and enter them as individual issues in JIRA. Things that will take weeks to be done should be there for us to measure the effort requires and assign deadlines. - do Django REST API **GET requests done** ## 2021-11-01 **Meeting - Harris:** - use _ModelViewSet_ in views.py instead of API getters - **DONE** - include POST requests in REST API - in order to harvest lectures from CDS - **DONE** - start testing (TDD) - **STARTED** **Helpful links:** - https://docs.djangoproject.com/en/3.2/topics/testing/overview/ - https://www.django-rest-framework.org/api-guide/testing/ ## 2021-11-02 **Meeting - Harris:** - finish testing the API - **DONE** - after the tests are done, add authentication to API - **Pending** **Helpful links:** - https://simpleisbetterthancomplex.com/tutorial/2018/11/22/how-to-implement-token-authentication-using-django-rest-framework.html ## 2021-11-03 **Meeting - Harris - in presence of Maria:** - make a mockup to discuss how to present mixed CDS results - mockup for PDF results - **Pending** - define test methods for the API: _GET /lectures_, _GET /lectures_, _POST /lecture_ - **DONE** ## 2021-11-04 **Meeting - Harris:** - fixed issues in API tests - **DONE** - start UI - fix issues with _yarn install_ - **DONE** ## 2021-11-05 **Meeting - Harris:** - use _ant.design_ UI Framework to make the site responsive - **PENDING REACT UDEMY COURSE** - implement the layout + menu bar + footer on the main page - **DONE** ## 2021-11-18 **Meeting - Harris:** - implement the main page + video page + search page - **DONE** - focus on minimal requirements - do not use too much styling for now ## 2021-11-30 **Meeting - Harris:** - implement navigation: https://reactrouter.com/docs/en/v6/getting-started/tutorial - **DONE** - grid of videos on the front page - **DONE** - display a lecture on the video page - **DONE** - display videos on the search page - **DONE** ## 2021-12-02 **Meeting - Harris:** - fix the layout on the search page - **DONE** - fix the layout on the video page - **DONE** - search button - missing search functionality - fix it - **DONE** ## 2021-12-09 **Meeting - Harris, Maria:** - due dates, [Board view](https://its.cern.ch/jira/secure/RapidBoard.jspa?rapidView=7494&amp;selectedIssue=ATWEB-4) on JIRA ## 2021-12-10 **Meeting - Harris:** - create the layout of the video page - **DONE** - finish the Footer - **DONE** - make everything responsive - **Pending** ## 2021-12-15 **Meeting - Maria:** - focus on the UDEMY courses until they expire (Dec-31) - **it has been extended until February 28 2022** - mixed results with PDFs - mockup is not necessary - it can be coded right away - find a way to display PDF readers on the website ## 2022-01-10 **Meeting - Harris:** - in the URL: 1. instead of _?filter_ - use _?queue_ - **DONE** 2. instead of _/results_ - use _/search_ - **DONE** - run the backend - **DONE** - don't forget to run the virtual environment! - finish the Footer (!!!) - **DONE** - fix the Search layout - **DONE** - the UI has to be as close to the desired design as possible before connecting it to the backend - only one record is available so far in the backend -> more records will be added by Harris ## 2022-01-18 **Meeting - Harris:** - clean up .col in css - use ant design cols instead - **DONE** - change the footer from regular cols to ant design cols - **DONE partly** - there is a small design issue to fix - make the enter button for the search bar functioning! - **Pending** - clean up attributes in tsx that are not used in css (like "grid-style") - **DONE** - but grid-style is actually used in css ## 2022-01-24 **Meeting - Harris:** - use Axios to retrieve data from the API to the UI: https://www.freecodecamp.org/news/how-to-use-axios-with-react/ - finish the video page first - **DONE** - you need a token from localhost/admin/authtoken/tokenproxy/ ## 2022-01-26 **Meeting - Harris:** - create a separate component for API root, then make an instance of it when you have to reach it - **DONE** - pass the parameter of lectureID in the API get request (on the video page) - **DONE** - fix the source of the iframe video (lectureID in the URL) - **DONE** Note: there are some CROS + authorization issues when retrieving data from the API for other pages (like search page) - fix this - **DONE** After installing `poetry django-cors-headers` on the server side, in the `settings.py` file (backend): ``` INSTALLED_APPS = [ ... "corsheaders", ] MIDDLEWARE = [ ... "corsheaders.middleware.CorsMiddleware", ] ``` ## 2022-01-27 **Meeting - Harris:** - connect the search page to the API - api/v1/search/lectures/?search=something - **Pending** - after the search page is done, we move to the main page to select the most recent videos - token is not needed for the UI (for now) ## 2022-01-31 **Meeting - Harris:** - 401 Unauthorized issue - **SOLVED** with `permission_classes = (IsAuthenticatedOrReadOnly)` in `views.py` file (cds backend) - other errors - fix the code - **DONE** 1. TypeError: Cannot read properties of undefined (reading 'toLowerCase') 2. TypeError: lectures.filter is not a function ## 2022-02-02 **Presentation for Harris' section of the Academic Training Website** **Meeting - Harris, Maria:** - Iterate through the Array from the API requested data (use loops for now): https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map - Rehearsals for the Terra Incognita presentation ## 2022-02-04 **Meeting - Harris:** - instead of `response.data` it supposed to be `response.data.results`, so now all the API calls work (search page + main page) - https://academictraining.cern.ch/ is deployed in the Kubernetes cluster - however, when we refresh the lecture page, it gives back 404 Not Found Request Code - fix some minor things in the UI, like changing regular containers to ant-design components (grid system is really important(!)) - resize to Mobile view - fix the Search function (it does not suppose to be visible in the URL when typing in search bar, only after submitting) - fix the Enter button in Search ## 2022-02-15 **Meeting - Harris, in presence of Salome Rohr:** Comments and ideas of **Salome**: - What is the website about? I think it is not clear for general public, people who do not know the academic training programme. I would suggest to add a paragraph on the homepage or a page 'about' in the main menu. - 'Most recent' section: video are not easily identifiable. I would suggest to display the full title or at least more words. The date could also be useful already at this level (without having to click on the video to see it.) - Search results: **the date of the video is also useful already at this level I think.** - Except the search, is there any other way to access the content? It looks, at first, that there is only the videos in the 'most recent' section. It is not clear to me that you can see more videos by searching. What about adding a menu with videos by subjects? It would mean indexing them by subjects if it is not yet the case. On CDS there is a list with subjects - **But for now this is not doable, because we don't have data of subject categories** - Footer: are all the links useful to the audience? I think about 'CERN Printshop', 'Writing guidelines', etc. - On the detailed lecture pages, there is an issue with the html tags that are displayed in the summary. I would also add the length of the video. I think it could be useful (It is personally a criteria when deciding to watch it now or later or for choosing a short/long video maybe also display it in the search results). Add also the link to the corresponding record on CDS. - Concerning the copyright, I think at the bottom of the website, it is the copyright for the website in general. I think about the videos themselves. E.g.: https://academictraining.cern.ch/lectures/2782493 who holds the copyright? what can I do or whom should I ask if I am e.g. a teacher and would like to show/use it during one of my courses, etc. I don't know what kind of agreement concerning copyright/license you have with the lecturer when registering a lecture and uploading it online. On videos.cern.ch, there is the copyright and link to the 'conditions of use' e.g: https://videos.cern.ch/record/2743984 - **other way to access content: browsing by Year.** - add a Search guide/tips: how to search in the title, how to search for a specific lecturer, how to search for a specific year/date, etc. - add a link to download the video (maybe a link redirecting to CDS). - add an information about slides generally available on the Indico page. - for the ones that are 'From the same series', find a way to indicate them/group them to be easily identifiable as such, e.g.: https://indico.cern.ch/event/1049663/ https://indico.cern.ch/event/1049666/ - add in the footer a link to the Indico category: https://indico.cern.ch/category/72/ and to the collection on CDS: https://cds.cern.ch/collection/Academic%20Training%20Lectures?ln=en Comments of **Harris**: - focus on About Us page - put dates in the search results - **finish the slides for the Terra Incognita presentation first (!)** - fix HTML attributes in the abstract: https://stackoverflow.com/questions/37337289/react-js-set-innerhtml-vs-dangerouslysetinnerhtml ## 2022-02-15 **Meeting - Harris, in presence of Maria:** - the Terra Incognita presentation will be only a reference at the ATC meeting (**March 15 - 9h30** - it will last only 15 min) - the Terra Incognita presentation should cover the technical details of the website - Franciska covers the Analysis, Requirements, Design part - Harris the Architecture - Project Future should be discussed with Maria(!) - About Us page - template **DONE** - finish the page, and add continuously the profiles of ATC members - the precision for the Terra Incognita agenda is needed - add Salome's comments in JIRA - and give updates for Salome if we covered something from her ideas/comments - usability tests for users - the sponsor in the AT lectures is needed, and it doesn't exist in teh CDS records - **SNOW Ticket needed** - that Maria will open ## 2022-02-22 **Terra Incognita rehearsal - with Harris and Maria:** - do not go to too much detail in the presentation - some changes must be made (like removing Comments from the Requirements) - change the Footer (!) ## 2022-02-28 **Terra Incognita Presentation** https://indico.cern.ch/event/1080998/ Recorded video included. ## 2022-03-02 **Meeting with Kasper and Maria:** - User Cases + User stories must be elaborated ASAP - Use Case diagrams + Sequence Diagrams must be made - Franciska must be involved more with the backend - not only frontend - disadvantage due to being ahead of university agenda/courses (like Single Page Applications - React) - how do we apply SCRUM - focus on the process methodology - create the Mark-down Documentation + apply it to the VIA BCs thesis template **Meeting Harris:** - Process Methodology what we have used so far: **Waterfall**, however we can change our method to **SCRUM** and do sprints - tasks from Kasper+Maria meeting has been shared with Harris - focus on the Group Description + "Group Contract" for the 1st Assignment - share it with Maria, Harris and Salome before handin - when the 1st Assignment is done, we can move on the code later - **fix the responsive design - MOBILE VIEW** ## 2022-03-03 **Group Description and Group Contract hand in at VIA** ## 2022-03-04 Other assignments at VIA - **Action for Franciska** DEADLINES: 1. PROJECT DESCRIPTION - DRAFT - **31.03.2022** 2. PROJECT DESCRIPTION - FINAL - **28.04.2022** 3. REQUIREMENTS/USER STORIES - **09.05.2022** All of them until **13:00**. Maria's input: - *if you do need to make reference to groups but you can change, please start searching for another kind of grouping. This is too loaded with stereotypes indeed.* During our Terra Incognita rehearsal, we talked about the "Contact Us" block in the footer - Franciska's questions were: 1. Question: a) What are the contact details of ATC? - **atc-members@cern.ch - this should be discussed on March 15th. I have already put it in https://indico.cern.ch/event/1130998/ If they prefer, we can put Academic-Training.Office@cern.ch, which is the administrator.** b) What is expected to be seen there? - **a web form that opens a mailto: the address decided in a. Maybe some text saying: Write to us your suggestions for future lecture series we could organise. Prepare it and show it at the meeting to collect comments.** c) If these contact details exist somewhere, how can I access them? - **you can open https://e-groups.cern.ch/e-groups/EgroupsSearchForm.do, put the names of the e-groups mentioned in a and see all the info.** 2. Question: Do we have to deploy a new version of the site with this page before the ATC meeting? - **Yes, please. We should see and discuss the page before going into the meeting.** ## 2022-03-07 **Repository of progress reports about the Academic Training Web site: https://indico.cern.ch/event/1136720/** ## 2022-03-09 **Mira Buzanszky** usability test - Comments: - tags, lables for search function - help to search things if there are no categories **"Not sure what to look for? Use these tags"** - be careful for the Font styles! - photos are too fast - change the titles in video lecture cards (at least 2 lines) - ideas for ATC members that do not want their pictures be displayed: https://biblio.vevey.ch/default/http-biblioveveych-portal-cmspageadminaspxp103.aspx ## 2022-03-15 **ATC Meeting:** - the fonts does not show the same way as in some browsers - make some more tests for different browsers - combine the AT header with the CERN toolbar after scrolling - **logo is not clearly visible**, give some contrast - is too discreet and very subtle - director's cut and academic hat is not clear - not very clear what is it about - **Contact Us**: generic list of emails can be displayed / archived - some filtering must be done before getting any spams from suggestions / emails - filtering what is spam and what is content - a new e-group must be made if any spams come in - search function must be improved - tags and lables might be useful - but then **keywords are needed**, however in the academic training submissions forms keywords are not available - there is no way to introduce keywords for now **Maria's notes from the meeting: https://indico.cern.ch/event/1130998/note/** ## 2022-03-29 **Maria Meeting:** - **focus on fixing the display on various browsers (Firefox!)** - **DONE** - put the About Us link to the top banner - **DONE** - update the ATC logo - **DONE** - make a script for fetching the sponsors from the records' Indico event - **Pending** - change the ATC description on the About Us page - **DONE** - create the documentation site academictraining-admin.docs.cern.ch - **Pending** - deploy the new version with the changes - **DONE** - put the ATC email address under the Contact Us link in the bottom banner - **DONE** In general fix other issues like: - change the current elements to ant.design components (most of them) - **DONE** - fix the mobile view for several devices - **DONE** - fix the preload GIF on the website before it loads the fonts (it doesn't work somehow) - **Pending** - just remove it! ## 2022-04-01 **Maria, Harris Meeting:** - various browsers - if on one browser the layout does not work, then it has to be changed completely - Harris said that he's pretty sure that it happens because of the custom styling - sponsors are not available in CDS - we don't know where it will be available on CDS (but it was promised in one of the SNOW tickets) - however, we don't need it from CDS - we can just retrieve the data from Indico - the SPONSOR data is stored in Indico's API as **"organizer"** ## 2022-04-08 **Maria, Kasper Meeting:** - change the **Project Description**, focus on the problem itself because it's not well explained - make another graph with another loop in the **Methodology** - describe roles from your team and how did we applied our methodology to our project - change the dates in the **Time Schedule** - "March-June 2023" is confusing and put 2 dates with the final deliverable product for CERN - June 2022 and final report - March 2023 ## 2022-04-12 **Maria, Harris Meeting:** - new deployment with the new top banner - write email to ATC-members with the updates - **DONE** - update mobile view for Search page - **DONE** - fix Drawer content bug in Header - **DONE** - add "No results Found" if there are no search results - **DONE** - add "Loading" components - **DONE** - fix arrows in Pagination - **DONE** - display number of Search results - **DONE** - remove extra padding from the top of videos when play button is on hover - **DONE** - change the animation for the member cards (About Us page) when on hover - **DONE** ## 2022-04-13 **Harris Meeting:** - search bar should be always open when it has content in it (especially when there are no results and new search is needed) - Harris suggested to remove the **Search results: "{searched content}"** from the Search page and replace it with the number of results ## 2022-04-19 **Maria Meeting:** - ask again Kasper about the Final Project Description - submit before April 22 - **DONE** - some changes were made for the Methodology - change Albert De Roeck to **André David** + his picture on the **ABOUT US** page - **DONE** - write an email Massimo Giovanozzi if he wants to display a picture of himself on the website or not - just get a confirmation - **DONE** - make a draft/a line for JIRA tickets from now until the end of May (it could be even a drawing) - **DONE** - Maria suggested to retrieve all the data from Indico API rather than CDS - because the sponsors will be retrieved from Indico anyways - make commits and deployments more often - try not to be dependent on Harris - the documentation site **academictraining-admin.docs.cern.ch** should be created **FOR** the maintainers ## 2022-04-20 **Harris, Maria Meeting:** - loading component added - Maria likes it, however Harris is not fun of it - **stakeholders' feedback is needed** - otherwise change it - Maria suggested to retrieve all the data from Indico API, however, according to Harris, we cannot abandon completely CDS because CDS has more lectures (including the PDFs) than Indico - Indico has only the events that were made for Academic Training lectures - remove the blank/white background from the clicked footer menu items (!) - **DONE** - finish the remained tasks for the website development before April 22 ## 2022-04-21 **Timeline:** | JIRA Ticket number | Task | Due date | | ------------------ | ------------------------------------------------------------------------------------------------------- | ------------ | | ATWEB-4 / 34 | Edit the logo | **03 May - DONE** | | ATWEB-41 | Fix arrows in Pagination | **21 April - DONE** | | ATWEB-37 | Update Mobile View for Search Page | **21 April - DONE** | | ATWEB-44 | Change member cards | **21 April - DONE** | | ATWEB-35 | **Create** the documentation site academictraining-admin.docs.cern.ch (simultaneously with VIA reports) | **31 May** | | ATWEB-30 / 31 | harvest the lecture series' Sponsor=Organizer in the Indico fields | **31 May - DONE** | | - | VIA Requirements/User stories | **06 May - DONE** | | ATWEB-47 | Update search functionality with the 'keywords' | **10 June** | | ATWEB-9 / 17 | Add CERN Oauth | **10 June** | | ATWEB-1 | script to fetch the "Number of Views" | **10 June** | | ATWEB-28 | Usability tests with users for the Academic Training web site | **20 July** | | ATWEB-27 | Write the academictraining-admin.docs.cern.ch in MkDocs | **17 June** | | ATWEB-12 | Make a mockup to discuss how to present mixed CDS results | **31 May** | ## 2022-05-03 - Rebase went wrong on Git - fix it - **DONE** - Put the "Contact-Us" next to the "ABOUT US" on the top banner - **DONE** - the Search button slightly bigger + **Search** text next to it - **DONE** - add the **_Duration_** to the videos; the MARCXML tag is 300 - **DONE** - there are two-channel videos now with Academic Training videos - it should be tested how do they look like on the website - **Pending** ## 2022-05-05 **Urs Wiedermann + Maria Dimou meeting:** - instead of "Indico", put "Full programme" - **Put Events instead, DONE** - "Events" should be placed on the header - separate page is not needed, it is enough to navigate to the site of Indico - the searchable fields must be updated!!! ATWEB-30 JIRA ticket - **DONE** - searching by speaker, sponsor etc. is definitely needed - search field must stay open if there are no search results - **DONE** - for example if "climate research" gets typed, Urs doesn't want to get everything from research - so the search might work only by words, not the full string - sponsors should give keywords to avoid searches with single words (ex. "beauty charm" not "beauty" AND "charm") - **DONE** - keywords must be taken from Indico API, because the sponsors have access only to that - not CDS - **DONE** - plan for adding "Favorites", "Share" and "Like" - chronological prioritization is needed on the Search page (search results should be indexed by most recent) - **DONE** ## 2022-05-05 **Harris, Maria and Salome Meeting at R1:** - mixed results with PDFs, default PDF thumbnail is needed - some lectures are also stored as TIFs, might worth to look into it - Salome looks into these records without videos (the ones with PDFs) - search bar should be visible permanently - **DONE** - the search bar also must be in the header and be displayed in the middle (open JIRA ticket) - **DONE** ## 2022-05-13 **Harris, Maria Meeting:** - make a connection with the Indico API and retrieve the data of sponsors and keywords - **DONE** - the CERN SSO login is too complicated to establish it until June 10, but saving _Favorites_ and _Likes_ could be possible locally - with CERN login it is not going to be guaranteed ## 2022-06-07 **ATC Meeting: - the boost destroys the order of the results (by date) - the lectures that are the most closest to the keywords come up first - so it is because of the score of the records - in Google results it does not show up the website - so it is pretty hard for users to find it - **Pending** - in Safari the layout is different and it must be changed ASAP - **Pending** Other useful feedback: "Pippa Wells to Everyone 09:16 AM It looks very nice. It would be good if the results of a search had choices for sort by date, or by speaker name, for example (I just tried searching for Higgs)" "From Pippa Wells to Everyone 09:19 AM Looks good for me in Firefox too It also seems that I can't search for a speaker, only for a talk title (I can go into "events" and use the indico search with all its features)" - logo sent out to ATC members ## 2022-06-13 **Harris, Maria Meeting:** - number of views - find a way to retrieve it from CDS - advertise the website in Mattermost channels - see what is priority and what is not and then we ask for feedback The requirements for search: - some lectures are not shown from CDS - gives an error message from CDS - ex. "x.400" - should return exact search results -should return similar results ## PRIORITY by the end of JULY: 1. create the doc site (!) - **Franciska** 2. add speaker to search - **Harris** 3. add sponsor to search - **Harris** 4. add a dropdown menu of relevance (Sort by: Most relevant (by score), Newest first (by date), Oldest first (by date)) in the Search results - **Harris** 5. older CDS records carry the `speaker` details in datafield tag `100` not `700` - **Harris** 6. add under in the help button how to search (the default is "AND" if you want to search by "OR", search like this etc.) - **Franciska** 7. add double quotes to find the exact match - **Harris** CERN SSO is dropped until Franciska leaves. ## 2022-07-17 - Kasper and Maria meeting - Supervision for the BSc Thesis Draft: - title not very accurate - developing a website for AT for example - must be accurate description ex. "Website for the etc" - **DONE** - Abbreviations in the beginning for example after the Table of Content - **DONE** - Rich picture to Introduction - for overview - **DONE** - some part of the Analysis should go to the Introduction - **DONE** - The last Note from Requirements can be removed ("The stakeholders etc..") - **DONE** - Use Case diagram can be reduced like just "view lecture" - keep only the blue bubbles - **DONE** - Use Case "Descriptions" not "Scenarios" - **DONE** - Base Sequences can be separated to 2 columns (User-Admin) - **Pending** - explain design decisions that were restricted - like disabling Comments because the site had to be public - **DONE** - Domain model is missing from the Design (!!!!) - very important - it belongs to the Analysis - it's also just a partial one - put a small arrow next to the actions - lecture_id can be excluded - **DONE** - huge gap in Design part - for example React architecture, alternatives **DONE**, design patterns used, all the structure and behavior, life cycles - **DONE**, state management, how to use Redux, data fetching, data models, structural diagram, server setup, data flow in react, components, tools, npm or yarn - **DONE**, axios - **DONE** etc - architecture is a must, set up modules, describe hooks, diagram with all the functions, sequence diagram fetching data from the backend, client-server side, webpacks if used etc - **Pending** - why it is mono-directional from CDS to our app - **Pending** - technologies why you've chosen one over another - **DONE** - more text in Implementation, for example useCallback hook why is there - explain more in the Implementation part - **Pending** - illustrate some features and forms - **Pending** - Navigation can go to Design (why React Router, what benefits you) - **DONE** - Tests are important - TDD - Unit tests etc. - **DONE** - Process Report write it early - **DONE** - 20-30 pages plus appendices - chop it at the end if it is needed - usabilities and feedback can be part of the Test and can be explained as such - **DONE** - next meeting for Process Report **16th of August 3PM** ## Maria's notes for the Documentation Web site 1. Can the file names in each directory be more tailored or just index.html? Now, one sees for example https://academictraining-admin.docs.cern.ch/setup/file.en.html - meaning _file.en.html_ in _every_ directory. A bit bogus... 2. Can we have a section in the doc, easy to find in the Contents, with the actual code github repo name and its owner(s)/maintainers/developers/users, the OKD4 project name and the gitlab repo name for the Markdown Documentation site and _who_ can edit, please? ## 2022-07-19 - Harris, Maria meeting - searches are somehow case-sensitive - they shouldn't be ex. https://academictraining.cern.ch/search/?search=Arsuaga-Rios&page=1 - make speakers non case-sensitive - remove "series" from the (?) help tooltip - **DONE** - change filenames in the doc - instead of file.en.md use index.md - **DONE** - change tooltip - **DONE** ## 2022-08-18 - Kasper, Maria meeting - Supervision for Process Report - "attractive" adj. can be ommitted from the headline / title, because it's subjective - some part of the Group Description must go to the Project Execution - Group Description in general can be updated, e.g. how did it influence the development working with other people at CERN - the Process Report is **too descriptive, not reflective** - it's okay to use informal language and 1st person - try to go into depth in the Project Execution - show all of your artifacts and backlogs - logbook does not have to be an Appendix (you can put it in the report) - Project Reflection must reflect to the project development, not only the internship - Additional Delimitations can be put into the Project Description under Delimitations (delete Appendix) - put State Management (like Redux) into the Project Report - Process Report shall be extended with the early mockups and the development of the completed view (can be mentioned in the Process Report and then put in the Appendix) ## 2022-09-12 - Maria meeting - Supervision for Process Report - make sure that there are no references to the **Additional delimitations** as an appendix, if it is merged to the Project Description - **DONE** - put same title in all reports - **DONE** - article 'a' in the title - **DONE** - put the **old mockups** in addition, why we could not put categories, keywords in the records - while our discovery during the development - **DONE** ## 2022-09-28 - Maria meeting - Supervision **Updated Project Description, Project & Process Report on Indico** - User Guide is still needed to be written into a document, for now it only exists on the AT documentation site - **Pending** - Project Report updated with Tests, Package Managers, Containerization + Navigation has been put under the Design - add missing Redux to Project Report - **Pending** - write email to Kasper if Appendices have to be attached to the end of the reports or not (to not have it replicated) - **Pending** - upload also the appendices to Indico - **Pending**