--- # Multi factor authentication deployment in IT and its effects outside IT --- ## Disclaimer * This presentation is only 15 minutes (questions included) * This slide deck contains many slides * How to get, register, use a token * Technical details of the implementation * Potential future steps * I will only present part of the slides today * Go through the slides at your own convenience * Please contact [us](https://cern.service-now.com/service-portal/report-ticket.do?name=computer-security&se=computer-security) if you have questions --- ## What is mFA/2FA? * Protecting authentication beyond your first factor * First factor stolen/phished: *no impact* * Can protect: * Single-Sign-On (SSO): any web app using the new SSO * SSH: any SLC6, CC7, Centos 8 system ---- ### Supported 2nd factors * Supported 2nd factors & requirements: * [SSO&SSH] TOTP: smartphone authenticator app * [SSH] Yubico OTP: Yubikey * [SSO] U2F/WebAuthN: physical token (e.g. Yubikey) * SMS not supported anymore: * Not supported natively by Keycloak * Not considered as secure as other options ---- ### How to register a 2nd factor * Registered at first use on the SSO (remotely): * TOTP (smartphone authenticator app) * U2F/WebAuthN (Yubikeys@SSO) * Registered remotely via dedicated website: * Yubico OTP (Yubikeys@SSH): https://sshsetup.web.cern.ch/ * No need to go to the self-service stations at CERN anymore ---- ### Web 2FA authentication  * Password + 2nd factor only * No step-up support, need to logout & re-login ---- ### SSH authentication  * (Password|Kerberos|SSH Key) + 2nd factor * Without kerberos, asking for login for unix service accounts * login must be in `.k5login`  * Protip: Yubikey OTP or TOTP can be entered directly ---- ### Obtaining a (recent) Yubikey * Relatively expensive tokens (~50$): * First batch discounted, but can't count on it * Privately bought Yubikeys not useable for SSH: * Keys need to be configured with keys known by CERN * Not in CERN stores: removed due to inactivity * Might be re-added in the future ---- ### Obtaining a Yubikey outside IT * CERN IT providing the keys (at cost): * Not tracked through Inventory: to be bought via TID * Let us know ASAP if you will need more that 15-20 keys * Contact us via [Service Now](https://cern.service-now.com/service-portal/report-ticket.do?name=computer-security&se=computer-security) * Please group request per section/group/experiment/... ---- ### Obtaining a Yubikey in IT * Distributed by the IT secretariat for IT (February 2020): * Asked back when leaving CERN (`circuit de départ`) * Special case: Summer students, COAS, ... * Their `circuit de départ` doesn't include IT secretariat * Asked/managed by and charged to the supervisor ---- #### Using U2F on Linux systems * Browsers need to access U2F devices: ACLs required * Transparent for recent (end 2019) systemd/udev * Dependencies automatically installed: * Fedora: `u2f-hidraw-policy` * Ubuntu: `libu2f-udev` * CERN Centos 7: * Need to install `u2f-hidraw-policy` * Package in EPEL, being added to CERN repo * Scientific Linux 6: * Firefox 68 seems incompatible with U2F... --- ## Implementing 2FA * New implementation very similar to previous one * Has been used by Computer Security team for years * Starting to be used by IT-CS & IT-DB * Tightly integrated into the new SSO * Yubikey OTP supported separately for now * Long term plan to integrate it tighter ---- ### Web 2FA authentication * Fully integrated into new standard SSO * Special option when defining roles  ---- ### SSH 2FA authentication * Configured via puppet's `multifactor` module * Support for new infra: [CRM-3404](https://its.cern.ch/jira/browse/CRM-3404) & [CRM-3420](https://its.cern.ch/jira/browse/CRM-3420) * Fully integrated in Centos 7 and 8 via pam: * Transparent for `ssh`, `scp`, `sftp`, `rsync`, ... * Degraded support on SLC6 (not recommended): * Not using pam, not transparent ---- ### Availability: #### Two independent backends * TOTP & U2F/WebAuthN: * Fully dependent on Keycloak * No other dependency * Yubikey OTPs/SSH: * User-yubikey mapping: Openshift + Postgresql DBOD * Yubikey validation: WebDFS + Oracle DB * To be re-evaluated on the long term (better intregration) ---- ### Availability: #### Emergency access * Even with redundancy, 2FA might fail * Important in particular for disaster recovery * Currently evaluating fail-over procedures for SSH * Most likely using SSH keys on hardware tokens * Solution to be put in place before general deployment ---- ### Previous infrastructure * SSH: * Puppet default configuration pointing to new infra * Old configuration to be removed from Puppet * SSO: * Few 2FA sites: to be migrated as soon as possible * Backend (opensts.cern.ch) retired afterwards * Self-registration stations for 2FA: * Will be removed in the very near future --- ## Deployment plan: ### Step 1 -- AIADM & Puppet core Target: March 31st 2020 * Protect AIADM with 2FA * Protect puppet core services with 2FA: * Foreman/Judy: website & API * Teigi applications (API): pwn, tbag, tellme * Mcollective ---- ### Affecting IT and many more! * Same infrastructure for IT and other users * Hard to have 2FA for IT only: complex filters * Limited benefit to only restrict IT: * Services could still be attacked through non-IT users * Expected only to impact *power-users* * Deploying/configuring Puppetized VMs and services * End-users (of services) should not be impacted ---- ### Protect puppet service APIs with 2FA * Can't modify the APIs themselves: * Using IP-base whitelists * Transparent for AIADM clusters * Other systems: require equivalent security levels: * List maintained by the Computer Security Team * Access protected by 2FA (any: SSH, web, tool, ...) * Detailed logs of logins & actions sent to Security Team * SSH almost covered by central monitoring * Other applications to be discussed ---- ### Timeline (Step 1) * February 2020: * Users obtain and configure tokens (Yubikeys/Auth App) * Identify servers for IP whitelisting: contact [us](https://cern.service-now.com/service-portal/report-ticket.do?name=computer-security&se=computer-security) * Validate/test 2FA deployments: * SSH: `aiadm-multi.cern.ch` * SSO: use `Two-factor authentication` options * March 2020: * Enable 2FA on AIADM (SSH) and Foreman (SSO) * Enable IP restrictions for all APIs * Ensure security level of servers in IP whitelist * Deadline: March 31st (not April 1st) --- ## Deployment plan: ### Step 2.A -- SSH access to IT internals Target: Summer 2020 * Protect access to IT internal systems with 2FA * **Only for admin access in IT, no effect for *standard* users** * User access to LXPLUS not affected * Requiring either direct 2FA or through 2FA bastion * Subject to validation & approval in April 2020 ---- ### SSH access to IT internal systems * Through 2FA bastions (AIADM or equivalent): * Single factor authentication on the final system * Detailed logs from the bastion (execlog & netlog) * Direct access to the server with 2FA: * Multifactor configured on the final system * Access logs of the system sent to Security Team ---- ### Puppet configuration * Using a `toggle`: * Parameter configurable via Hiera * Added to `base` module (included everywhere) * Options: * `nothing`: Not adding any code * `multifactor`: Multifactor configured with IP whitelist * `ipfilter`: IP filter for 2FA bastions only ---- ### Proposed timeline (Step 2.A) * April 2020: * Review of project & approval of next steps * Add new `toggle` to `base` module (default: `nothing`) * May 2020 * Service managers in IT encouraged to change toggle and configure whitelists (bastions, internal connections, ...) * [Optional] Service managers ∉ IT to hardcode `nothing` * June 2020: * Set default to `multifactor` (CC7&C8), IP filter (SLC6): * `nothing` still acceptable outside IT --- ## Deployment plan: ### Step 2.B -- Provisioning services Target: 2020 * Subject to further review & approval * Targeted services: * Openstack for IT tenants * Gitlab for IT users (puppet & packages) * Koji ---- ### Openstack for IT tenants * Needs to be protected: * Access to destructive actions * Access to console: reboot + single-user mode * Pending integration of CLI with OIDC * Actual implementation to be discussed afterwards ---- ### Gitlab for IT users * Needs to be protected: * Puppet configuration * Package sources * 2FA already deployed but not fully supported * Completely disconnected from SSO's 2FA * Major bypass through direct git access * Depending on upstream: hard to improve... * Possibility for puppet (for shared modules & IT hostgroups): * Forbids pushing onto `master` branches * Require 2FA for all users on targeted repositories ---- ### Koji * Needs to be protected: * Packages installed & run * First step: enforce precise ACLs for each and every tag * 2FA restrictions difficult: Gitlab-CI integration... ---- ### Proposed timeline (Step 2.B) * No clear timeline: most problems still open * Targeting 2020 * Subject to review after Step 1 & 2.A --- ## Deployment plan: ### Step 3 -- Other admin accesses in IT * Any administrative accesses outside SSH * Anything else, any protocol * Service managers (in IT) should ensure: * Privileged/administrative access protected by 2FA * Access & action logs stored outside of the server * Think about it while adapting to new SSO/group/ldap! --- ## Actions/Timelines ### For people affected ---- ### Get your 2nd factor #### Only if you need one * Register TOTP on new SSO * [Optional] Obtain a Yubikey: * ∉ IT: Contact [us](https://cern.service-now.com/service-portal/report-ticket.do?name=computer-security&se=computer-security) to get keys (grouped by exp., group, ...) * Provided at cost (TID) * ∈ IT: Go to the IT Secretariat * Register it on new [SSO](https://auth.cern.ch) & https://sshsetup.web.cern.ch/ * Test 2FA Authentication: * On new [SSO](https://auth.cern.ch) * On `aiadm-multi.cern.ch` ---- ### Projected timeline * February 2020: * Contact [us](https://cern.service-now.com/service-portal/report-ticket.do?name=computer-security&se=computer-security) if using Foreman/tbag/teigi APIs * March 31st, 2020: * 2FA enabled on AIADM (SSH) * 2FA enabled on Foreman (SSO) * Foreman/Teigi/Tbag/Pwn/Tellme APIs restricted ---- ### Projected timeline #### Pending validation and approval * May 2020: * [TBC] Set *toggle* in `base` to `nothing` (outside IT) * June 2020: * [TBC] 2FA or IP restriction for admin access within IT --- ## Questions? --- ## More information * Computer Security web page on 2FA: * [English](https://security.web.cern.ch/security/recommendations/en/2FA.shtml) * [French](https://security.web.cern.ch/security/recommendations/fr/2FA.shtml) * Previous presentations (similar to this one): * [ASDF](https://indico.cern.ch/event/876460/) (with minutes of the discussions) * [Experiment coordination meetings](https://codimd.web.cern.ch/p/HJsCvfnW8) ---