Maria's notes for Theo's MSc thesis
===
**Actions and tips for the final MSc report**
1. Mention [the project](https://it-student-projects.web.cern.ch/projects/cern-solid-server-hosting) and structure your report based on the 1,2,3,... in there.
2. Check [Jan's MSc](https://cds.cern.ch/record/2771156?ln=en) to see length and format that itu.dk requires for MSc reports.
Open the editor and start writing the MSc sections, already.
2. Check [Aristofanis's BSc](https://cds.cern.ch/record/2724152?ln=en) to read the web app vs CERN SSO integration (page 27?).
* You can email Aristofanis for clarifications. He is in Copenhagen now. I shall put you in touch with an email. He has no more his CERN login.
* You can send a Direct Message (DM) in mattermost to CERN SSO expert Asier Aguado Corman. I'll put you in touch via mattermost.
3. Check [Lukas's MSc thesis](https://libstore.ugent.be/fulltxt/RUG01/002/494/871/RUG01-002494871_2018_0001_AC.pdf). It dates since 2018, which makes it useful because you can write in the thesis what changed since.
4. Mention the above 3 reports in the references, to the extent that you'll use them.
5. Open https://paas.cern.ch to **Create new web site**, with your CERN login. Give name **cern-solid** to the web site. Select option **PaaS**! Please read [the PaaS CERN doc](https://paas.docs.cern.ch/) before starting.
* You can send a Direct Message (DM) in mattermost to CERN Web expert Michal Kolodziejski about PaaS questions and OpenShift. asking. I'll put you in touch via mattermost.
* You can open a request to the PaaS support team [via this form](https://cern.service-now.com/service-portal?id=sc_cat_item&name=request&se=Paas-Web-App)
4. Once the installation is done, please ping Maria to get a pod. Then we shall do the campaign to get others joining. **Guest** accounts are possible for getting large numbers of testers. See image of any CERN app that invokes the SSO:
The CERN SSO uses OAuth2 or OpenID Connect, which are web standards for authentication and work very well for web applications. This is a good place to start: https://auth.docs.cern.ch/user-documentation/oidc/oidc/
Digita.ai sent the document https://docs.digita.ai/dgt-id-broker/providers/keycloak.html on keycloak and CSS. To investigate.
5. Write in the Thesis report the installation adventures with CERN PaaS and Intranet vs Internet access restrictions, as per [this discussion](https://mattermost.web.cern.ch/it-dep/pl/pyidezgp67nfmfpbxp31yg1kxy)
## Dates for plans
1. Theo to email Sebastian and suggest he joins our Fri 15 October 10h00 Zoom.
2. CERN is closed for year end shutdown 22/12-4/1. We cannot get any expert advice or users on board during that time. All must be complete by then, only thesis report writing is possible.
## Thesis structure
To be discussed with Sebastian, this is Maria's proposal. Sebastian says 40-50 pages _of quality_ is OK. [itu.dk guidelines](https://itustudent.itu.dk/Study-Administration/Project-Work/Workload-and-Project-Size ). The report will be in [CDS](https://cds.cern.ch), so it should contain useful lessons from the project. Here are the chapters:
1. Introduction (contains [project definition and reference](https://it-student-projects.web.cern.ch/projects/cern-solid-server-hosting))
2. What is Solid
* Sources: Two White Areas - find them in http://solid.cern.ch AND Jan's thesis (see link above) AND Lukas's thesis (see link above) AND https://solidproject.org
4. CERN IT environment - a few words
* Describe the experience of working with Asier, Michal and Maria. Explain the CERN SSO set-up, Openshift (=OKD4) and PaaS.
6. The CSS internals and software status
7. Discussions from gitter channels
https://gitter.im/solid/test-suite,
https://gitter.im/cern-solid/community and
https://gitter.im/solid/community-server
showing the (non?)commitment of the CSS development community. We can ask for a meeting with them, if you wish. Let's discuss it...
6. Discussion on the CERN-required Auth protocols and the effort to equip CSS with them.
7. Instructions for the UI, what will people use.
8. How the testers' community was approached.
9. How performance was measured.
10. Conclusions and Recommendations
## Technical questions how much storage would you need?
1. What is the problem to define https://cern.ch/cern-solid - if defined via https://cern.ch/paas then
2. First build a docker image to deploy in OKD4.
3. Should CSS run as a VM in OpenStack or as a container in OKD4? OKD4 is most suitable for web apps.
4. How much storage would we need to host as many CSS pods as the people who have a CERN login (let's say a round number of 10K users).
5. Is there a CSS default pod storage space?
6. Are there any databases needed to run a CSS Solid server ? Possible but for now just text files' storage. For ex. Indico profiles in tables in a SQL db. SSO uses MySQL.
7. Digita.ai is a company contributing keycloak IDP to CSS: https://docs.digita.ai/dgt-id-broker/providers/keycloak.html uses a proxy to wrap Solid WebIDs and present them as OpenIDs. **Why they chose the _proxy_ option instead of implementing _OpenID, OAuth2 or SAML_. Important to understand licensing issues related to cost. Also need to have a copy of the code in order to audit. Is this usable by the CERN instance?** Can it be subject to security issues that we need to check? How? Submit to the security team. Chk what Michal wrote in mattermost about CodiMD. The app portal handles the AuTH. Asier sent https://github.com/solid/solid-auth-oidc to check if it can be used.
8. For testing purposes create _guest_ accounts as per the screenshot above - also Google accounts can be used. Permissions are limited. No risk for security. https://account.cern.ch
## Screenshots to crop and use in the thesis
## Maria's CSS usage notes
1. Ugly WebID https://community-server-testbase2.app.cern.ch/dimou/profile/card#me - to ask any CERN user to try it, we should eventually define the PaaS web server at CERN that would enable more elegant WebIDs, namely
https://cern-solid.cern.ch/dimou/profile/card#me.
*I have created https://cern-test-css.app.cern.ch I couldn't get a shorter url name. Will ask Michal if I can get a shorter URL*
2. Never managed to open the site from outside CERN, although the _tunneling_ command I use opens other restricted sites.
```sshuttle --dns -r dimou@lxplus.cern.ch 10.0.0.0/8 10.100.0.0/16 10.254.0.0/16 10.76.0.0/15 100.64.0.0/10 128.141.0.0/16 128.142.0.0/16 137.138.0.0/16 172.16.0.0/12 185.249.56.0/22 188.184.0.0/15 192.168.0.0/16 192.65.196.0/23 192.91.242.0/24 194.12.128.0/18 -x 188.184.98.218 -x 172.17.0.2 -x 172.17.0.3 -x 172.17.0.4 -x 192.168.0.2```
*In the new instance at https://cern-test-css.app.cern.ch this issue should be fix. But the website is now open to the world instead of just CERN's users. It imply more responsability toward security.*
3. When one tries to login to CSS with an existing WebID, one is given 3 strings to enter in the previous pod to prove one's own identity. It remained impossible to find **where** these 3 strings should be entered either with the [penny interface](https://penny.vincenttunru.com/explore/?url=https%3A%2F%2Fdimou.solidcommunity.net%2Fprofile%2Fcard%23me#https%3A%2F%2Fdimou.solidcommunity.net%2Fprofile%2Fcard%23me) or with the [solidcommunity.net one](https://dimou.solidcommunity.net/profile/card#me).
*Should be removed in new instance. It is removed for testing purpose only, a alternative solution should be found to keep the security token, one solution could be to improve the current interaction with a "seamless" one. See https://github.com/solid/community-server/discussions/1006*
4. Even after Theo said that he disabled the step requiring this matching between the CSS and a previously existing WebID, the error still occurs:
*Should be removed in new instance.*
From [this issue discussion](https://github.com/solid/community-server/discussions/1006) one understands that there is a patch that can be used for a _test_ instance but not for a _production_ one. What does it mean? That we have to tell our users to forget about their previous pods and start from scratch?
*"Production" is a term designing the real life environement of an application or software, consumed by "real" users, as opposed to a "testing" environement.*
> That we have to tell our users to forget about their previous pods and start from scratch?
*Let's talk about this in zoom*
5. The entry page of CSS, when used with the right ```https://``` prefix, contains a link to a _userguide_, which doesn't exist.
*I Couldn't not find the deadlink*
6. Overall, it is impossible to get any substantial amount of users to try the server, get a pod, give us comments and generate enough traffic to make performance measurements.
*Given that the issue from point 1,2,3 and 4 have been fixed, should this be possible now?*
7. Maria asked Lukas (point 4 at the beginning) to try the CERN CSS test instance. Reply:
_I tried creating a pod on the CSS you sent, using my own WebID (https://lukkie.be/profile/card#me). It says that this was successful, but I am unable to use it to log in on e.g. https://penny.vincenttunru.com/explore/?url=https%3A%2F%2Fcommunity-server-testbase2.app.cern.ch
Also, after creating my account, I can re-create it immediately, indicating that probably nothing was created...
I don't think you can have multiple pods on a CSS (and if you do, it's not clear how). I am honestly not sure what the benefit is of registering to a different CSS using my own, existing WebID and without creating a new pod (see screenshot). I guess it would allow you to use the login portal of this CSS, but why would I not use my own login portal? It's really unclear to me.
By the way, I was unable to recover my old WebID unfortunately. It got lost when DigitalOcean had an incident which caused many of their droplets to be deleted, including mine. Fortunately, I wasn't actively using it anymore.
Anyway, as for my own WebID, I have set it up today using the most recent version of CSS and I was able to add some information using Mashlib. Incredible that the UI is still the same as in 2017, and it's really bad, just like you mentioned on Tuesday. I also logged in on Penny (https://penny.vincenttunru.com/explore/?url=https%3A%2F%2Flukkie.be%2Fprofile%2Fcard) which works fine. It's a way better UI, but it's still super user-unfriendly. I also tried logging in to various other apps found on https://solidproject.org/apps such as for instance Poddit: https://vincenttunru.gitlab.io/poddit/
But for all of the apps I tried, the log-in widget is not able to connect to both my pod, and the pod you linked me (see second screenshot).
I think the big problem with Solid at the moment is that it's just super user-unfriendly. Creating a friendly UI on top of this would indeed be a very big step forward._
Indeed, earlier today Maria wrote the following in the SolidOS channel:
8. Indeed, we should understand and explain in the thesis what the point is to "link" pod logins via an existing WebID, especially because of [this mattermost post](https://mattermost.web.cern.ch/it-dep/pl/efs3mhzybigcpmqy18n7tnm4by). It doesn't work for Maria, it works for Theo, it can only be kept for the test built?!?!?
*Yes, I was able to reproduce Lukas' bug and I understand where it comes from. Thank you and Lukas for reporting this. Will try to fix it in the next days.*
## 2021-10-27
1. CSS WebID for user ```dimou``` https://cern-test-css.app.cern.ch/dimou/profile/card#me shows nothing in the browser and opens a separate text file that looks like this:
```
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix solid: <http://www.w3.org/ns/solid/terms#>.
<>
a foaf:PersonalProfileDocument;
foaf:maker <https://cern-test-css.app.cern.ch/dimou/profile/card#me>;
foaf:primaryTopic <https://cern-test-css.app.cern.ch/dimou/profile/card#me>.
<https://cern-test-css.app.cern.ch/dimou/profile/card#me>
solid:oidcIssuer <https://cern-test-css.app.cern.ch/>;
a foaf:Person.
```
Wherelse dimou's WebID on solidcommunity.net https://dimou.solidcommunity.net/profile/card#me) shows the whole profile
Understood that solidcommunity.net (NSS flavour) is part of SolidOS and has a (not so good) UI, wherelse CSS is just a "pod providing Solid server" with nothing else.
Nevertheless, **what one can expect from the CERN CSS instance has to be written down in the thesis.**
Also, as discussed today:
* where are the pod files in CERN storage
* how to keep the instance up and running
* pass web server ownership to user 'dimou'
* how to make more user friendly prompts on login. Examples:
One clicks on https://cern-test-css.app.cern.ch/dimou and gets:
But one has already "been signed up"! So, explain on the prompts the difference between "Sign-up" and "login".
## Presentations
Maria invites Theo to give a **Terra incognita lecture at CERN on January 31st at 15h00 CET** https://indico.cern.ch/event/1092335/ (Theo given full Indico management rights on the event) after the ITU MSc committee defense. This is a totally informal meeting on Zoom. About 25 people join from our group.
Please see here all related info:
1. Lecture index https://indico.cern.ch/category/11108/
2. Terra incognita definition https://twiki.cern.ch/LCG/WhiteAreas
3. CERN-Solid PoC talk https://indico.cern.ch/event/1031678/
4. CERN-Solid introductory talk https://indico.cern.ch/event/979381/
Maria also suggests that Theo speaks at **the January 13th at 16h00 CET Solid World**. This is how to apply https://es1cz4pb7oi.typeform.com/to/nietD34f
Decided on 2021/12/16 to move to February (is it 10th or 17th)?
## Agreed on 2021/11/17 within Sebastian, Theo, Maria
Agreement
deployment of CSS version with UI:
20211118 instance with UI, but w/o secure token
20211124 instance w secure token
minimal user guide
## 2021/11/24 meeting with Sebastian, Theo, Maria
On CERN test CSS instance https://cern-test-css.app.cern.ch/dimou/profile/
I still can do nothing as I am prompted to login, although not clear how/where.
On issue https://github.com/solid/community-server/discussions/1006 the outcome is not clear. If we offer the CERN test instance without CERN SSO integration we **Must** have the selected CSS configuration on https://gitlab.cern.ch/ in order to ask the security experts here for a code review.
## 2021/12/06 Maria's notes from a CS3 conference meeting
Please do **not** remove. They can't be on the public index http://solid.cern.ch for "political" reasons:
Discussing the status of open source Solid servers' status that can be defended today:
```
The Community Solid Server (CSS) was announced (as v.1) in August 2021. Maria has WebID https://cern-test-css.app.cern.ch/dimou/profile/card#me on the CERN test instance. However, no real use is possible because of:
a. error to access one's own profile.
b. lack of UI.
c. no ID provider (integration with the CERN SSO not yet done).
Other issues related to CSS today:
a. Web Access Control (WAC) specification violation found by the test-suite. This can lead to inter-operability problems with other Solid servers.
b. Minor security considerations for some operations. Namely, with some Access Control options CSS allows someone other than the pod owner allowed to edit a file can also to delete it.
We can, if needed, consider an evaluation of the Solid-NextCloud (SolidServer php) in the Policy document for a Solid server at CERN before concluding on the preferred Solid server implementation for CERN. **However** With CernBOX moving from PHP (OC10) to Go (OCIS), the option of using a php-based solid server makes less sense. It would have made sense if there was a path towards putting it into CernBOX, but that path seems to be closed now...
```
In particular in the CERN LIVE-IT meeting of 2021/12/17 https://indico.cern.ch/e/1088962 the future of CERNBox seems obscure.
## 2021/12/16 meeting with Theo, Maria
Email from Theo to Wouter Termont digita.ai developer on the keycloak proxy:
```
Hello Wouter,Now I'm entering the last month of my thesis and will focus on the writing part. I can not give much time anymore for the proxy installation.From my experience, the proxy is still buggy and heavily lacks a user manual that I didn't find in the current doc ( I think the current explain how things work whereas I was looking for how to make it work).When shipping the proxy to CERN or another client, I think they should be able to install and maintain it independently. I have familiarity with npm, an understanding of Solid-OIDC, and experiences with difficult installations. I don't think that it should require more than that to make the proxy work.For now, I don't believe the installation nor the maintenance can be done without relying strongly on the Digita team. Therefore I cannot recommend the use of the proxy to CERN in its current status. But I am aware that the proxy is still a work in progress and won't forget to mention it as well.Still, it would be very exciting to see CERN users log in to their Pod with CERN SSO. But with the time remaining, I would be willing to give it another try only if I can have a good manual. By the for former, I mean:
the manual should only assume that the user has a CSS and a keycloak instance running
the manual should explain how to setup keycloak, how to install the proxy and set up the proxy with baby steps, in the manner of a Lego manual
It should end up with a successful authentication to CSS with a keycloak user.
I can see that a branch is working on a keycloak guide, so maybe some work has already been done in that direction, what is the status on that?
```
## CSS instance at CERN
https://css.app.cern.ch/dimou/
Guide the user in the first page - Please explain to the user that verification should be done only once and the Write permission should be put in the original pod.
