[flp@j183 ~]$ o2-flp-setup deploy --head j183.localdomain --flps j183.localdomain Could not copy SSH key. You will be prompted for the password for user root. SSH password: Executing playbook flp-multinode.yml - all on hosts: all - Gathering Facts... j183.localdomain ok Ensure pciutils are installed... j183.localdomain ok Gather CRU PCI addresses... j183.localdomain ok Register number of CRUs found... j183.localdomain ok Gather CRORC PCI addresses... j183.localdomain ok Register number of CRORCs found... j183.localdomain ok Define number of Hugepages for CRU deployments... j183.localdomain skipped Define number of Hugepages for CRORC deployments... j183.localdomain skipped Gather head node facts when head is excluded... j183.localdomain skipped j183.localdomain skipped Set hostname of central services based on Ansible facts... j183.localdomain ok Check if /etc/o2.d/FLP_suite_version exist... j183.localdomain ok Update available disk size needed... j183.localdomain ok Ensure that free space on "/" is greater than 2.5 GB... j183.localdomain ok: { "changed": false, "msg": "All assertions passed" } Make sure we remove any control packages from old versions of FLP suite... j183.localdomain done Make sure we remove any configuration packages from old versions of FLP suite... j183.localdomain done Make sure we remove any dataflow packages from old versions of FLP suite... j183.localdomain done Make sure we remove any monitoring packages from old versions of FLP suite... j183.localdomain done Make sure we remove any qc packages from old versions of FLP suite... j183.localdomain ok Ensure EPEL repository present... j183.localdomain ok Ensure CERN repository exists... j183.localdomain ok Check if ali02-c7-base.repo is present... j183.localdomain ok Ensure ali02-c7-base.repo is absent... j183.localdomain skipped Ensure CentOS-Base repository is present... j183.localdomain skipped Ensure CentOS-Updates repository is present... j183.localdomain skipped Set up ALICE Software EL7 repo for non-updatable RPMs (GPN)... j183.localdomain ok Set up ALICE O2/FLP CC7 RPM repo (GPN)... j183.localdomain done Ensure clean yum cache... j183.localdomain done | stdout: Loaded plugins: changelog, fastestmirror, kernel-module, langpacks, protectbase, : tsflags, versionlock Cleaning repos: CentOS-CERN alice-ctp3 alice-o2-cc7 alice-system-deps alisw-el7 : base centos-sclo-rh centos-sclo-rh-source : centos-sclo-rh-testing-source centos-sclo-sclo : centos-sclo-sclo-source centos-sclo-sclo-testing-source cern : elrepo epel extras ipbus-sw-base ipbus-sw-updates mesosphere : mesosphere-noarch nodesource pgdg-common pgdg10 pgdg11 pgdg12 : pgdg13 pgdg95 pgdg96 updates Cleaning up list of fastest mirrors Remove sendmail forwarding rule for root... j183.localdomain ok Make sure O2 configuration dir exists... j183.localdomain done Make sure O2 writeable configuration dir exists... j183.localdomain ok Disable SELinux... j183.localdomain ok Write FLP suite version... j183.localdomain done Write FLPs inventory... j183.localdomain done Set up ALICE O2 CC7 RPM repo... j183.localdomain done Make sure the InfoLogger package is installed... j183.localdomain ok Ensure configuration dir exists... j183.localdomain ok Make sure the net-tools package is installed... j183.localdomain ok Deploy infoLoggerD service override... j183.localdomain ok Deploy infoLoggerD configuration... j183.localdomain done Ensure InfoLoggerD runs at boot time... j183.localdomain ok Install xorg... j183.localdomain ok Ensure environment-modules is at the latest version... j183.localdomain ok Ensure perf is installed... j183.localdomain ok Install packages needed for LXC guest... j183.localdomain skipped Enable firewalld in LXC guest... j183.localdomain skipped Install NTP... j183.localdomain ok Configure NTP... j183.localdomain done Enable NTP client... j183.localdomain done Restart o2-infologger-daemon (via handler)... j183.localdomain done - head on hosts: head - Gathering Facts... j183.localdomain ok Ensure pciutils are installed... j183.localdomain ok Gather CRU PCI addresses... j183.localdomain ok Register number of CRUs found... j183.localdomain ok Gather CRORC PCI addresses... j183.localdomain ok Register number of CRORCs found... j183.localdomain ok Define number of Hugepages for CRU deployments... j183.localdomain skipped Define number of Hugepages for CRORC deployments... j183.localdomain skipped Gather head node facts when head is excluded... j183.localdomain skipped j183.localdomain skipped Set hostname of central services based on Ansible facts... j183.localdomain ok Ensure postgresql-11 is installed... j183.localdomain ok Init postgresql-11... j183.localdomain ok | stdout: skipped, since /var/lib/pgsql/11/data/log exists Ensure postgresql-11 service runs immediately and on startup... j183.localdomain ok Populate service facts... j183.localdomain ok Make sure local CCDB is stopped and does not get loaded at reboot... j183.localdomain skipped Uninstall systemd script for local CCDB (if it was there at all)... j183.localdomain skipped Deploy pg_hba.conf for external access... j183.localdomain ok Prepare database for CCDB... j183.localdomain ok | stdout: GRANT CREATE EXTENSION ALTER ROLE | stderr: NOTICE: extension "hstore" already exists, skipping Create installation directory... j183.localdomain ok Download CCDB... j183.localdomain ok | msg: OK (16009792 bytes) Install Java... j183.localdomain ok Install config.properties... j183.localdomain ok Install systemd script... j183.localdomain ok Firewall - Open CCDB port... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation Start CCDB on reboot of the machine... j183.localdomain ok Ensure configuration dir /etc/flp.d/ccdb-sql exists... j183.localdomain ok Deploy configuration file for repository cleaner... j183.localdomain ok Ensure that the repocleaner runs every 10 minutes.... j183.localdomain ok Ensure EPEL repository present... j183.localdomain ok Ensure CERN repository exists... j183.localdomain ok Check if ali02-c7-base.repo is present... j183.localdomain ok Ensure ali02-c7-base.repo is absent... j183.localdomain skipped Ensure CentOS-Base repository is present... j183.localdomain skipped Ensure CentOS-Updates repository is present... j183.localdomain skipped Set up ALICE Software EL7 repo for non-updatable RPMs (GPN)... j183.localdomain ok Set up ALICE O2/FLP CC7 RPM repo (GPN)... j183.localdomain done Ensure clean yum cache... j183.localdomain done | stdout: Loaded plugins: changelog, fastestmirror, kernel-module, langpacks, protectbase, : tsflags, versionlock Cleaning repos: CentOS-CERN alice-ctp3 alice-o2-cc7 alice-system-deps alisw-el7 : base centos-sclo-rh centos-sclo-rh-source : centos-sclo-rh-testing-source centos-sclo-sclo : centos-sclo-sclo-source centos-sclo-sclo-testing-source cern : elrepo epel extras ipbus-sw-base ipbus-sw-updates mesosphere : mesosphere-noarch nodesource pgdg-common pgdg10 pgdg11 pgdg12 : pgdg13 pgdg95 pgdg96 updates Cleaning up list of fastest mirrors Remove sendmail forwarding rule for root... j183.localdomain ok Make sure O2 configuration dir exists... j183.localdomain done Make sure O2 writeable configuration dir exists... j183.localdomain ok Disable SELinux... j183.localdomain ok Write FLP suite version... j183.localdomain done Write FLPs inventory... j183.localdomain done Download Consul v1.8.3 from https://alice-daq-yum-o2.web.cern.ch/download/consul_1.8.3_linux_amd64.zip... j183.localdomain ok | msg: OK (42054014 bytes) Install unzip... j183.localdomain ok Unzip Consul... j183.localdomain ok Create consul group... j183.localdomain ok Create consul user... j183.localdomain ok Create directories... j183.localdomain done | item: /etc/consul/consul.d j183.localdomain done | item: /var/lib/consul j183.localdomain done Install binary file... j183.localdomain ok Install systemd script... j183.localdomain ok Install telemetry config file... j183.localdomain done Cleanup local... j183.localdomain ok j183.localdomain ok j183.localdomain ok Add firewall exceptions for Consul... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok Ensure Consul is started at boot time... j183.localdomain ok Ensure Consul is running... j183.localdomain ok Ensure mariadb is installed... j183.localdomain ok Ensure mariadb service runs immediately and on startup... j183.localdomain ok Ensure MySQL-python is installed... j183.localdomain ok Ensure database ports are open... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok Set root user password... j183.localdomain ok | msg: User unchanged Set up ALICE O2 CC7 RPM repo... j183.localdomain done Make sure the InfoLogger package is installed... j183.localdomain ok Ensure configuration dir exists... j183.localdomain ok Make sure the net-tools package is installed... j183.localdomain ok Execute infoLogger DB config script... j183.localdomain ok | stdout: Running non-interactive, automated mode selected Setting up mysql for infoLogger Database INFOLOGGER already exists, skipping database create Can already connect as infoLoggerServer, skipping user create Can already connect as infoBrowser, skipping user create Can already connect as infoLoggerAdmin, skipping user create MySQL infoLogger accounts created No changes debug... j183.localdomain ok: { "changed": false, "msg": { "changed": false, "cmd": [ "sh", "/opt/o2-InfoLogger/bin/o2-infologger-newdb", "-i", "-s", "\n SQL_ROOT_PWD=\n INFOLOGGER_DB_NAME=INFOLOGGER\n INFOLOGGER_DB_HOST=j183\n EXTRA_USER[server]=infoLoggerServer\n EXTRA_USER[browser]=infoBrowser\n EXTRA_USER[admin]=infoLoggerAdmin\n EXTRA_PWD[server]=ilPwdS\n EXTRA_PWD[browser]=ilPwdB\n EXTRA_PWD[admin]=ilPwdA\n INFOLOGGER_CONFIG=\n" ], "delta": "0:00:00.257980", "end": "2021-04-20 11:37:47.325626", "failed": false, "rc": 0, "start": "2021-04-20 11:37:47.067646", "stderr": "", "stderr_lines": [], "stdout": "Running non-interactive, automated mode selected\nSetting up mysql for infoLogger\nDatabase INFOLOGGER already exists, skipping database create\nCan already connect as infoLoggerServer, skipping user create\nCan already connect as infoBrowser, skipping user create\nCan already connect as infoLoggerAdmin, skipping user create\nMySQL infoLogger accounts created\nNo changes", "stdout_lines": [ "Running non-interactive, automated mode selected", "Setting up mysql for infoLogger", "Database INFOLOGGER already exists, skipping database create", "Can already connect as infoLoggerServer, skipping user create", "Can already connect as infoBrowser, skipping user create", "Can already connect as infoLoggerAdmin, skipping user create", "MySQL infoLogger accounts created", "No changes" ] } } Deploy InfoLogger configuration file... j183.localdomain done Ensure the message table exists... j183.localdomain ok | msg: non-zero return code | stdout: infoLoggerAdminDB Database connected Creating infoLogger table | stderr: Error - Failed to execute create table messages(severity char(1), level tinyint unsigned, timestamp double(16,6), hostname varchar(32), rolename varchar(32), pid mediumint unsigned, username varchar(32), `system` varchar(32), facility varchar(32), detector varchar(32), `partition` varchar(32), run int unsigned, errcode int unsigned, errline smallint unsigned, errsource varchar(32), message text, index ix_severity(severity), index ix_level(level), index ix_timestamp(timestamp), index ix_hostname(hostname(14)), index ix_rolename(rolename(20)), index ix_system(`system`(3)), index ix_facility(facility(20)), index ix_detector(detector(8)), index ix_partition(`partition`(10)), index ix_run(run), index ix_errcode(errcode), index ix_errline(errline), index ix_errsource(errsource(20))) Table 'messages' already exists Ensure InfoLoggerServer runs at boot time... j183.localdomain ok Ensure InfoLogger database ports are open... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok Install NodeJS... j183.localdomain ok Configure npm proxy... j183.localdomain skipped Add new user 'gui'... j183.localdomain ok Create installation directory... j183.localdomain ok Install InfoLogger-GUI from npm-registry 1.4.2... j183.localdomain done Install config file for InfoLogger-GUI... j183.localdomain done Create o2-ilg directory in /var/lib if it does not exist... j183.localdomain ok Create DB JSON file for ILG Profiles... j183.localdomain done Install systemd config... j183.localdomain ok Firewall - Open HTTP/HTTPS port... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation Firewall - Open HTTPS port... j183.localdomain skipped Start InfoLogger-GUI on reboot of the machine... j183.localdomain ok Set hostname for control mesos based on Ansible facts... j183.localdomain ok Install Mesos Logrotate configuration... j183.localdomain ok Install Mesos Logrotate service... j183.localdomain ok Install O2 apricot... j183.localdomain done Create apricot group... j183.localdomain ok Create apricot user... j183.localdomain ok Ensure apricot working directory path exists... j183.localdomain ok Open port 47101 for apricot... j183.localdomain ok | msg: Permanent operation Set up systemd unit for O2 apricot... j183.localdomain ok Start apricot... j183.localdomain ok Install Java runtime... j183.localdomain ok j183.localdomain skipped j183.localdomain skipped j183.localdomain ok included: /home/flp/.local/share/o2-flp-setup/system-configuration/ansible/roles/ansible-zookeeper/tasks/RedHat.yml for j183.localdomain Install OS Packages... j183.localdomain ok j183.localdomain ok included: /home/flp/.local/share/o2-flp-setup/system-configuration/ansible/roles/ansible-zookeeper/tasks/tarball.yml for j183.localdomain Ensure the tarball dir exists at /opt/src... j183.localdomain ok Ensure the zookeeper dir exists at /opt/zookeeper-3.4.14... j183.localdomain ok check whether the tarball is downloaded already... j183.localdomain ok Download zookeeper v3.4.14 from https://alice-daq-yum-o2.web.cern.ch/download/zookeeper-3.4.14.tar.gz... j183.localdomain ok | msg: HTTP Error 304: Not Modified Unpack tarball.... j183.localdomain ok | stdout: skipped, since /opt/zookeeper-3.4.14/bin exists group... j183.localdomain ok user... j183.localdomain ok Change ownership on zookeeper directory.... j183.localdomain ok Create zookeeper {{ item }} directory.... j183.localdomain ok j183.localdomain ok j183.localdomain ok Write myid file.... j183.localdomain ok Configure zookeeper zoo.cfg... j183.localdomain ok Add zookeeper's bin dir to the PATH... j183.localdomain skipped j183.localdomain ok included: /home/flp/.local/share/o2-flp-setup/system-configuration/ansible/roles/ansible-zookeeper/tasks/upstart.yml for j183.localdomain Check if /etc/init exists... j183.localdomain ok Upstart script.... j183.localdomain skipped j183.localdomain ok included: /home/flp/.local/share/o2-flp-setup/system-configuration/ansible/roles/ansible-zookeeper/tasks/systemd.yml for j183.localdomain Systemd script.... j183.localdomain ok Firewall - Open port... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok included: /home/flp/.local/share/o2-flp-setup/system-configuration/ansible/roles/ansible-zookeeper/tasks/common-config.yml for j183.localdomain Configure zookeeper-env.sh... j183.localdomain skipped Update the log4j config with saner production values... j183.localdomain ok Enable zookeeper service... j183.localdomain ok Set mesos_install_mode to 'master-slave' if j183.localdomain is both head and flp... j183.localdomain ok Set mesos_install_mode to 'master' if j183.localdomain is head node only... j183.localdomain skipped j183.localdomain ok debug... j183.localdomain ok: { "changed": false, "msg": "Install Mode is master-slave" } Add apt-key... j183.localdomain skipped Add mesosphere repo... j183.localdomain skipped Pin Mesos version... j183.localdomain skipped Install Debian OS packages... j183.localdomain skipped Install os packages... j183.localdomain ok create work dir... j183.localdomain ok Remove Default Mesos Configs... j183.localdomain ok Remove Mesos ZK File... j183.localdomain ok Remove Default Mesos-Master Config... j183.localdomain done Remove Default Mesos-Slave Config... j183.localdomain done Mesos default config file... j183.localdomain ok Mesos Master config file... j183.localdomain ok Mesos Slave config file... j183.localdomain done create /etc/mesos-master dir... j183.localdomain done create /etc/mesos-slave dir... j183.localdomain done Ansible create /etc/mesos-master/logging_level... j183.localdomain done Ansible create /etc/mesos-slave/logging_level... j183.localdomain done Ansible create /etc/mesos-slave/work_dir... j183.localdomain done Mesos master upstart script... j183.localdomain skipped Mesos slave upstart script... j183.localdomain skipped Check if systemd exists... j183.localdomain ok Add script to check infiniband interfaces for Mesos Slave... j183.localdomain ok Mesos master systemd script... j183.localdomain ok Mesos slave systemd script... j183.localdomain ok Enable/Disable mesos-master service... j183.localdomain ok Enable/Disable mesos-slave service... j183.localdomain ok Install coconut... j183.localdomain done Template AliECS configuration... j183.localdomain ok Make sure component aliecs exists... j183.localdomain done | stdout: Success! Data written to: o2/components/aliecs/ Deploy AliECS configuration... Entry updated: settings Configuration imported: aliecs/ANY/any/settings | stderr: Loading coconut/v0.22.2-1 Loading requirement: BASE/1.0 Install AliECS core... j183.localdomain ok Configure firewall for Mesos... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok Start ZooKeeper... j183.localdomain ok Create AliECS group... j183.localdomain ok Create AliECS user... j183.localdomain ok Ensure AliECS core configuration path exists... j183.localdomain ok Set up systemd unit for AliECS core... j183.localdomain ok Systemctl daemon-reload... j183.localdomain ok Open port 47102 for AliECS... j183.localdomain ok | msg: Permanent operation Ensure AliECS working directory path exists... j183.localdomain ok Ensure AliECS configuration repositories path exists... j183.localdomain ok Ensure nginx 1.16.1 is present... j183.localdomain ok Ensures /var/www/html dir exists... j183.localdomain ok Copy main page... j183.localdomain done Deploy Nginx configuration file... j183.localdomain ok Deploy main configuration file... j183.localdomain ok Start nginx on boot... j183.localdomain ok Open port 80 in firewall... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation Add user 'gui'... j183.localdomain ok Create installation directory... j183.localdomain ok Install Control-GUI from npm-registry 1.19.0... j183.localdomain done Execute npm rebuild in case NodeJS was updated... j183.localdomain skipped Install config file for Control-GUI... j183.localdomain done Install systemd config... j183.localdomain ok Firewall - Open HTTP port... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation Firewall - Open HTTPS port... j183.localdomain skipped Start Control-GUI on reboot of the machine... j183.localdomain ok Download Consul v1.8.3 from https://alice-daq-yum-o2.web.cern.ch/download/consul_1.8.3_linux_amd64.zip... j183.localdomain skipped Install unzip... j183.localdomain skipped Unzip Consul... j183.localdomain skipped Create consul group... j183.localdomain skipped Create consul user... j183.localdomain skipped Install binary file... j183.localdomain skipped Get run number from consul... j183.localdomain failed | msg: non-zero return code | stderr: Error! No key exists at: o2/aliecs/run_number Update run number in o2/runtime/aliecs/run_number... j183.localdomain skipped Delete obsolete consul path... j183.localdomain skipped Add control defaults to consul... j183.localdomain done | item: monitoring_readout_url | stdout: Success! Data written to: o2/runtime/aliecs/defaults/monitoring_readout_url j183.localdomain done | item: monitoring_qc_url | stdout: Success! Data written to: o2/runtime/aliecs/defaults/monitoring_qc_url j183.localdomain done | item: monitoring_dd_url | stdout: Success! Data written to: o2/runtime/aliecs/defaults/monitoring_dd_url j183.localdomain done | item: monitoring_dpl_url | stdout: Success! Data written to: o2/runtime/aliecs/defaults/monitoring_dpl_url j183.localdomain done | item: infologger_server_host | stdout: Success! Data written to: o2/runtime/aliecs/defaults/infologger_server_host j183.localdomain done Add control vars to consul... j183.localdomain done | item: monitoring_readout_url | stdout: Success! Data written to: o2/runtime/aliecs/vars/monitoring_readout_url j183.localdomain done | item: monitoring_qc_url | stdout: Success! Data written to: o2/runtime/aliecs/vars/monitoring_qc_url j183.localdomain done | item: monitoring_dd_url | stdout: Success! Data written to: o2/runtime/aliecs/vars/monitoring_dd_url j183.localdomain done | item: monitoring_dpl_url | stdout: Success! Data written to: o2/runtime/aliecs/vars/monitoring_dpl_url j183.localdomain done Store inventory_head to consul... j183.localdomain done | stdout: Success! Data written to: o2/runtime/aliecs/defaults/inventory_head Store inventory_flps to consul... j183.localdomain done | stdout: Success! Data written to: o2/runtime/aliecs/defaults/inventory_flps Store template defaults to consul... j183.localdomain done | item: modulepath | stdout: Success! Data written to: o2/runtime/aliecs/defaults/modulepath j183.localdomain done | item: infologger_severity | stdout: Success! Data written to: o2/runtime/aliecs/defaults/infologger_severity j183.localdomain done | item: infologger_mode | stdout: Success! Data written to: o2/runtime/aliecs/defaults/infologger_mode j183.localdomain done Add or update control default consul values... j183.localdomain done | item: {'key': 'o2/runtime/aliecs/default_revision', 'value': 'flp-suite-v0.17.0'} | stdout: Success! Data written to: o2/runtime/aliecs/default_revision j183.localdomain done | item: {'key': 'o2/runtime/aliecs/default_revisions', 'value': {'github.com/AliceO2Group/ControlWorkflows/': 'flp-suite-v0.17.0'}} | stdout: Success! Data written to: o2/runtime/aliecs/default_revisions j183.localdomain done Create consul o2 components... j183.localdomain done | item: o2/components/qc/ | stdout: Success! Data written to: o2/components/qc/ j183.localdomain done | item: o2/components/readout/ | stdout: Success! Data written to: o2/components/readout/ j183.localdomain done Check if custom configuration exists... j183.localdomain skipped Set file_details... j183.localdomain skipped Check if custom configuration exists in templates... j183.localdomain ok Set file_details... j183.localdomain ok Make sure configuration dirs exist... j183.localdomain skipped j183.localdomain skipped j183.localdomain skipped Check if coconut exist... j183.localdomain ok create template for each host... j183.localdomain skipped Delete old configuration files to consul... j183.localdomain skipped Store configuration files to consul... j183.localdomain skipped Ensure configuration dirs exist... j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok Deploy configuration file stfb-qc.dpl.json... j183.localdomain ok Deploy configuration file readout-qc.dpl.json... j183.localdomain ok Deploy configuration file stfb-tof.dpl.json... j183.localdomain ok Deploy configuration file minimal-dpl.dpl.json... j183.localdomain ok Populate service facts... j183.localdomain ok Make sure local CCDB is stopped and does not get loaded at reboot... j183.localdomain skipped Uninstall systemd script for local CCDB (if it was there at all)... j183.localdomain skipped Deploy pg_hba.conf for external access... j183.localdomain ok Prepare database for CCDB... j183.localdomain ok | stdout: GRANT CREATE EXTENSION ALTER ROLE | stderr: NOTICE: extension "hstore" already exists, skipping Create installation directory... j183.localdomain ok Download CCDB... j183.localdomain ok | msg: OK (16009792 bytes) Install Java... j183.localdomain ok Install config.properties... j183.localdomain ok Install systemd script... j183.localdomain ok Firewall - Open CCDB port... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation Start CCDB on reboot of the machine... j183.localdomain ok Ensure configuration dir /etc/flp.d/ccdb-sql exists... j183.localdomain ok Deploy configuration file for repository cleaner... j183.localdomain ok Ensure that the repocleaner runs every 10 minutes.... j183.localdomain ok Add user 'gui'... j183.localdomain ok Create installation directory... j183.localdomain ok Install QC-GUI from npm-registry 2.2.1... j183.localdomain done Execute npm rebuild in case NodeJS was updated... j183.localdomain skipped Install systemd config... j183.localdomain ok Firewall - Open HTTP/HTTPS port... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation Install config file... j183.localdomain done Create o2-qcg directory in /var/lib if it does not exist... j183.localdomain ok Create DB JSON file for QCG Layouts... j183.localdomain ok Start QC-GUI on reboot of the machine... j183.localdomain ok Ensure influxdb 1.8.2 is present... j183.localdomain ok Deploy main configuration file... j183.localdomain ok Create InfluxDB custom directory structure... j183.localdomain ok Firewall - Open HTTP/HTTPS port... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation Check that the private key and certificate files are present... j183.localdomain skipped Ensure InfluxDB is started on boot... j183.localdomain ok Wait for InfluxDB to respond on port 8086... j183.localdomain ok Set empty fact influxdb_ssl_string... j183.localdomain ok Set influxdb_ssl_string based on HTTPS flag... j183.localdomain skipped Verify whether InfluxDB is already configured... j183.localdomain ok | stdout: name: databases name ---- telegraf readout qc kafka distribution _internal cru crorc dpl Create admin account... j183.localdomain skipped Create database using custom credentials... j183.localdomain done | item: telegraf j183.localdomain done | item: readout j183.localdomain done | item: qc j183.localdomain done | item: kafka j183.localdomain done | item: distribution j183.localdomain done | item: cru j183.localdomain done | item: crorc j183.localdomain done | item: dpl j183.localdomain done Create read only user for Grafana... j183.localdomain skipped Grant permission to Grafana user... j183.localdomain done | item: telegraf j183.localdomain done | item: readout j183.localdomain done | item: qc j183.localdomain done | item: kafka j183.localdomain done | item: distribution j183.localdomain done | item: cru j183.localdomain done | item: crorc j183.localdomain done | item: dpl j183.localdomain done Create an alias to access the database locally without password... j183.localdomain ok Ensure grafana 7.4.3 is present... j183.localdomain ok Add the user 'grafana'... j183.localdomain ok Deploy main configuration file... j183.localdomain ok Create datasources directory... j183.localdomain ok Define InfluxDB protocol... j183.localdomain skipped Deploy InfluxDB data sources... j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok Create dashboards directory... j183.localdomain ok Define dashboard provisioning... j183.localdomain ok Create dashboards directory... j183.localdomain ok j183.localdomain ok j183.localdomain ok Deploy dashboards... j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok Deploy TPC dashboards... j183.localdomain ok j183.localdomain ok j183.localdomain ok Deploy AliECS dashboards... j183.localdomain ok j183.localdomain ok Open HTTP port in firewall... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation Ensure grafana is started at boot time... j183.localdomain ok Wait for grafana to respond on port 3000... j183.localdomain ok Adjust number of DIM servers... j183.localdomain ok Open ports 5100-* for DIM Servers... j183.localdomain ok | msg: Permanent operation j183.localdomain ok Install DIM RPM... j183.localdomain ok Open port 2505 for the DIM Nameserver... j183.localdomain ok | msg: Permanent operation Open port 2500 for DIM WebDID... j183.localdomain ok | msg: Permanent operation Deploy DIM DNS systemd unit... j183.localdomain ok j183.localdomain ok Deploy WebDID systemd unit... j183.localdomain ok j183.localdomain ok Enable o2-dim-dns.service... j183.localdomain ok Enable o2-dim-webdid.service... j183.localdomain ok Ensure telegraf 1.17.3 is present... j183.localdomain ok Detect whether ReadoutCard is installed... j183.localdomain ok | msg: non-zero return code Set ReadoutCard version... j183.localdomain skipped Retrieve InfluxDBv2 token... j183.localdomain skipped Set InfluxDB v2 token variable... j183.localdomain skipped Deploy main configuration file... j183.localdomain done Creates scripts directory... j183.localdomain ok Deploy CRU monitoring script... j183.localdomain skipped Deploy IB monitoring script... j183.localdomain ok Add telegraf user to pda group... j183.localdomain skipped Ensure telegraf is enabled... j183.localdomain ok Make sure unix socket has correct permissions... j183.localdomain ok Ensure telegraf is started on boot... j183.localdomain ok Add new user 'gui'... j183.localdomain ok Create Bookkeeping installation directory... j183.localdomain ok Install Bookkeeping from npm-registry 0.16.7... j183.localdomain ok Create Bookkeeping database... [WARNING]: Module did not set no_log for unsafe_login_password j183.localdomain ok Create user for Bookkeeping database... j183.localdomain ok | msg: User unchanged Create environment file for Bookkeeping... j183.localdomain done Install systemd config... j183.localdomain ok Firewall - Open HTTP/HTTPS port... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation Start Bookkeeping... j183.localdomain ok Restart consul (via handler)... j183.localdomain done Restart o2-infologger-server (via handler)... j183.localdomain done Restart o2-ilg (via handler)... j183.localdomain done Restart Mesos master (via handler)... j183.localdomain done Restart AliECS core (via handler)... j183.localdomain done Restart o2-cog (via handler)... j183.localdomain done Restart o2-qcg (via handler)... j183.localdomain done Restart telegraf (via handler)... j183.localdomain done Cleanup meta/slaves/latest (via handler)... j183.localdomain done Restart mesos-master (via handler)... j183.localdomain done Restart mesos-slave (via handler)... j183.localdomain done - flps on hosts: flps - Gathering Facts... j183.localdomain ok Ensure pciutils are installed... j183.localdomain ok Gather CRU PCI addresses... j183.localdomain ok Register number of CRUs found... j183.localdomain ok Gather CRORC PCI addresses... j183.localdomain ok Register number of CRORCs found... j183.localdomain ok Define number of Hugepages for CRU deployments... j183.localdomain skipped Define number of Hugepages for CRORC deployments... j183.localdomain skipped Gather head node facts when head is excluded... j183.localdomain skipped j183.localdomain skipped Set hostname of central services based on Ansible facts... j183.localdomain ok Ensure EPEL repository present... j183.localdomain ok Ensure CERN repository exists... j183.localdomain ok Check if ali02-c7-base.repo is present... j183.localdomain ok Ensure ali02-c7-base.repo is absent... j183.localdomain skipped Ensure CentOS-Base repository is present... j183.localdomain skipped Ensure CentOS-Updates repository is present... j183.localdomain skipped Set up ALICE Software EL7 repo for non-updatable RPMs (GPN)... j183.localdomain ok Set up ALICE O2/FLP CC7 RPM repo (GPN)... j183.localdomain done Ensure clean yum cache... j183.localdomain done | stdout: Loaded plugins: changelog, fastestmirror, kernel-module, langpacks, protectbase, : tsflags, versionlock Cleaning repos: CentOS-CERN alice-ctp3 alice-o2-cc7 alice-system-deps alisw-el7 : base centos-sclo-rh centos-sclo-rh-source : centos-sclo-rh-testing-source centos-sclo-sclo : centos-sclo-sclo-source centos-sclo-sclo-testing-source cern : elrepo epel extras ipbus-sw-base ipbus-sw-updates mesosphere : mesosphere-noarch nodesource pgdg-common pgdg10 pgdg11 pgdg12 : pgdg13 pgdg95 pgdg96 updates Cleaning up list of fastest mirrors Remove sendmail forwarding rule for root... j183.localdomain ok Make sure O2 configuration dir exists... j183.localdomain done Make sure O2 writeable configuration dir exists... j183.localdomain ok Disable SELinux... j183.localdomain ok Write FLP suite version... j183.localdomain done Write FLPs inventory... j183.localdomain done Set hostname for control mesos based on Ansible facts... j183.localdomain ok Install Mesos Logrotate configuration... j183.localdomain ok Install Mesos Logrotate service... j183.localdomain ok Install Control Core Coconut and OCC... j183.localdomain done Set mesos_install_mode to 'master-slave' if j183.localdomain is both head and flp... j183.localdomain ok Set mesos_install_mode to 'slave' if j183.localdomain is flp node only... j183.localdomain skipped j183.localdomain ok debug... j183.localdomain ok: { "changed": false, "msg": "Install Mode is master-slave" } Add apt-key... j183.localdomain skipped Add mesosphere repo... j183.localdomain skipped Pin Mesos version... j183.localdomain skipped Install Debian OS packages... j183.localdomain skipped Install os packages... j183.localdomain ok create work dir... j183.localdomain ok Remove Default Mesos Configs... j183.localdomain ok Remove Mesos ZK File... j183.localdomain ok Remove Default Mesos-Master Config... j183.localdomain done Remove Default Mesos-Slave Config... j183.localdomain done Mesos default config file... j183.localdomain ok Mesos Master config file... j183.localdomain ok Mesos Slave config file... j183.localdomain done create /etc/mesos-master dir... j183.localdomain done create /etc/mesos-slave dir... j183.localdomain done Ansible create /etc/mesos-master/logging_level... j183.localdomain done Ansible create /etc/mesos-slave/logging_level... j183.localdomain done Ansible create /etc/mesos-slave/work_dir... j183.localdomain done Mesos master upstart script... j183.localdomain skipped Mesos slave upstart script... j183.localdomain skipped Check if systemd exists... j183.localdomain ok Add script to check infiniband interfaces for Mesos Slave... j183.localdomain ok Mesos master systemd script... j183.localdomain ok Mesos slave systemd script... j183.localdomain ok Enable/Disable mesos-master service... j183.localdomain ok Enable/Disable mesos-slave service... j183.localdomain ok Configure firewall for Mesos... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation j183.localdomain ok Clear agent ID... j183.localdomain skipped Start Mesos slave... j183.localdomain ok Open port 50051 for odc-shim... j183.localdomain ok | msg: Permanent operation Download Consul v1.8.3 from https://alice-daq-yum-o2.web.cern.ch/download/consul_1.8.3_linux_amd64.zip... j183.localdomain skipped Install unzip... j183.localdomain skipped Unzip Consul... j183.localdomain skipped Create consul group... j183.localdomain skipped Create consul user... j183.localdomain skipped Install binary file... j183.localdomain skipped Get run number from consul... j183.localdomain failed | msg: non-zero return code | stderr: Error! No key exists at: o2/aliecs/run_number Update run number in o2/runtime/aliecs/run_number... j183.localdomain skipped Delete obsolete consul path... j183.localdomain skipped Add control defaults to consul... j183.localdomain done | item: monitoring_readout_url | stdout: Success! Data written to: o2/runtime/aliecs/defaults/monitoring_readout_url j183.localdomain done | item: monitoring_qc_url | stdout: Success! Data written to: o2/runtime/aliecs/defaults/monitoring_qc_url j183.localdomain done | item: monitoring_dd_url | stdout: Success! Data written to: o2/runtime/aliecs/defaults/monitoring_dd_url j183.localdomain done | item: monitoring_dpl_url | stdout: Success! Data written to: o2/runtime/aliecs/defaults/monitoring_dpl_url j183.localdomain done | item: infologger_server_host | stdout: Success! Data written to: o2/runtime/aliecs/defaults/infologger_server_host j183.localdomain done Add control vars to consul... j183.localdomain done | item: monitoring_readout_url | stdout: Success! Data written to: o2/runtime/aliecs/vars/monitoring_readout_url j183.localdomain done | item: monitoring_qc_url | stdout: Success! Data written to: o2/runtime/aliecs/vars/monitoring_qc_url j183.localdomain done | item: monitoring_dd_url | stdout: Success! Data written to: o2/runtime/aliecs/vars/monitoring_dd_url j183.localdomain done | item: monitoring_dpl_url | stdout: Success! Data written to: o2/runtime/aliecs/vars/monitoring_dpl_url j183.localdomain done Store inventory_head to consul... j183.localdomain done | stdout: Success! Data written to: o2/runtime/aliecs/defaults/inventory_head Store inventory_flps to consul... j183.localdomain done | stdout: Success! Data written to: o2/runtime/aliecs/defaults/inventory_flps Store template defaults to consul... j183.localdomain done | item: modulepath | stdout: Success! Data written to: o2/runtime/aliecs/defaults/modulepath j183.localdomain done | item: infologger_severity | stdout: Success! Data written to: o2/runtime/aliecs/defaults/infologger_severity j183.localdomain done | item: infologger_mode | stdout: Success! Data written to: o2/runtime/aliecs/defaults/infologger_mode j183.localdomain done Add or update control default consul values... j183.localdomain done | item: {'key': 'o2/runtime/aliecs/default_revision', 'value': 'flp-suite-v0.17.0'} | stdout: Success! Data written to: o2/runtime/aliecs/default_revision j183.localdomain done | item: {'key': 'o2/runtime/aliecs/default_revisions', 'value': {'github.com/AliceO2Group/ControlWorkflows/': 'flp-suite-v0.17.0'}} | stdout: Success! Data written to: o2/runtime/aliecs/default_revisions j183.localdomain done Create consul o2 components... j183.localdomain done | item: o2/components/qc/ | stdout: Success! Data written to: o2/components/qc/ j183.localdomain done | item: o2/components/readout/ | stdout: Success! Data written to: o2/components/readout/ j183.localdomain done Check if custom configuration exists... j183.localdomain skipped Set file_details... j183.localdomain skipped Check if custom configuration exists in templates... j183.localdomain ok Set file_details... j183.localdomain ok Make sure configuration dirs exist... j183.localdomain skipped j183.localdomain skipped j183.localdomain skipped Check if coconut exist... j183.localdomain ok create template for each host... j183.localdomain skipped Delete old configuration files to consul... j183.localdomain skipped Store configuration files to consul... j183.localdomain skipped Ensure configuration dirs exist... j183.localdomain ok j183.localdomain ok j183.localdomain ok j183.localdomain ok Deploy configuration file stfb-qc.dpl.json... j183.localdomain ok Deploy configuration file readout-qc.dpl.json... j183.localdomain ok Deploy configuration file stfb-tof.dpl.json... j183.localdomain ok Deploy configuration file minimal-dpl.dpl.json... j183.localdomain ok Ensure running kernel headers and sources are installed for DKMS... j183.localdomain ok Ensure relevant utils are installed... j183.localdomain ok Ensure pda-kadapter-dkms is installed (usually a couple of minutes)... j183.localdomain ok Install ReadoutCard RPM... j183.localdomain done Ensure configuration dir exists... j183.localdomain ok Add IOMMU support in the kernel command line... j183.localdomain ok Disable GHES through the kernel command line... j183.localdomain ok Write grub cfg file... j183.localdomain done | stderr: Generating grub configuration file ... Found linux image: /boot/vmlinuz-3.10.0-1160.24.1.el7.x86_64 Found initrd image: /boot/initramfs-3.10.0-1160.24.1.el7.x86_64.img Found linux image: /boot/vmlinuz-3.10.0-1160.21.1.el7.x86_64 Found initrd image: /boot/initramfs-3.10.0-1160.21.1.el7.x86_64.img Found linux image: /boot/vmlinuz-3.10.0-1160.15.2.el7.x86_64 Found initrd image: /boot/initramfs-3.10.0-1160.15.2.el7.x86_64.img Found linux image: /boot/vmlinuz-3.10.0-1160.11.1.el7.x86_64 Found initrd image: /boot/initramfs-3.10.0-1160.11.1.el7.x86_64.img Found linux image: /boot/vmlinuz-3.10.0-1160.6.1.el7.x86_64 Found initrd image: /boot/initramfs-3.10.0-1160.6.1.el7.x86_64.img Found linux image: /boot/vmlinuz-0-rescue-06bc5d324c0a4e1d81d750e2ff98eb19 Found initrd image: /boot/initramfs-0-rescue-06bc5d324c0a4e1d81d750e2ff98eb19.img done Check for hard memlock limit for pda users... j183.localdomain ok | stdout: @pda hard memlock unlimited Replace hard memlock limit for pda users... j183.localdomain ok Increase hard memlock limit for pda users... j183.localdomain skipped Check for soft memlock limit for pdausers... j183.localdomain ok | stdout: @pda soft memlock unlimited Replace soft memlock limit for all users... j183.localdomain ok Increase soft memlock limit for all users... j183.localdomain skipped Deploy hugepage configuration files... j183.localdomain ok j183.localdomain ok j183.localdomain ok Ensure group pda exists... j183.localdomain ok Get users... j183.localdomain ok Create flp user... j183.localdomain skipped Add flp to wheel and pda groups... j183.localdomain ok Install driver module... j183.localdomain ok Ensure driver module is inserted on boot... j183.localdomain ok Deploy o2-roc-hugepages-startup script used by the o2-roc-hugepages-config systemd unit... j183.localdomain ok j183.localdomain ok Deploy hugepages systemd unit... j183.localdomain ok j183.localdomain ok Deploy cru_template.cfg config file to be used by the o2-roc-config-startup script... j183.localdomain done Deploy o2-roc-config-startup script used by the o2-roc-config systemd unit... j183.localdomain ok j183.localdomain ok Deploy o2-roc-config systemd unit... j183.localdomain ok j183.localdomain ok Enable o2-roc-hugepages-config service... j183.localdomain ok Enable o2-roc-config service... j183.localdomain skipped Disable potentially present legacy services... j183.localdomain ok j183.localdomain ok j183.localdomain ok cru-fw-service... j183.localdomain ok Ensure workspace dir exists... j183.localdomain ok Ensure log dir exists... j183.localdomain ok Deploy rescan script... j183.localdomain ok j183.localdomain ok Deploy check script... j183.localdomain ok j183.localdomain ok Deploy CRU FW rescan systemd unit... j183.localdomain ok j183.localdomain ok Adjust number of DIM servers... j183.localdomain ok Open ports 5100-* for DIM Servers... j183.localdomain ok | msg: Permanent operation j183.localdomain ok | msg: Permanent operation j183.localdomain ok Install DIM RPM... j183.localdomain ok Open port 2505 for the DIM Nameserver... j183.localdomain ok | msg: Permanent operation Open port 2500 for DIM WebDID... j183.localdomain ok | msg: Permanent operation Deploy DIM DNS systemd unit... j183.localdomain ok j183.localdomain ok Deploy WebDID systemd unit... j183.localdomain ok j183.localdomain ok Enable o2-dim-dns.service... j183.localdomain ok Enable o2-dim-webdid.service... j183.localdomain ok Install ALF RPM... j183.localdomain done Deploy ALF systemd unit... j183.localdomain ok j183.localdomain ok Enable o2-alf service... j183.localdomain ok Make sure tcllib is installed... j183.localdomain ok Make sure the Readout package is installed... j183.localdomain done Ensure configuration dir exists... j183.localdomain ok Install base configuration file for readout with emulated input... j183.localdomain done Install STFB-enabled configuration file for readout with emulated input... j183.localdomain done Check if coconut exist... j183.localdomain ok Delete old readout configuration files to consul... j183.localdomain done | stdout: Success! Deleted key: o2/components/readout/readout-standalone-j183 Success! Deleted key: o2/components/readout/readout-stfb-j183 Success! Deleted key: o2/components/readout/readout-cru-j183 Store readout configuration files to consul... Entry updated: readout-standalone-j183 Configuration imported: readout/ANY/any/readout-standalone-j183 Entry updated: readout-stfb-j183 Configuration imported: readout/ANY/any/readout-stfb-j183 | stderr: Loading coconut/v0.22.2-1 Loading requirement: BASE/1.0 Make sure the DataDistribution package is installed... j183.localdomain done Make sure the QualityControl package is installed... j183.localdomain done Ensure configuration dir exists... j183.localdomain ok Deploy configuration file for readout qc... j183.localdomain ok Deploy configuration file for postprocessing qc... j183.localdomain ok Deploy configuration file stfb_to_daqtask.json... j183.localdomain ok Deploy configuration file hmpid_raw_qc.json... j183.localdomain ok Deploy configuration file mft-digits-qc.json... j183.localdomain ok Deploy configuration file mft-raw-qc.json... j183.localdomain ok Check if coconut exist... j183.localdomain ok Delete old qc configuration files to consul... j183.localdomain done | item: /etc/flp.d/qc/readout_to_daqtask.json | stdout: Success! Deleted key: o2/components/qc/readout_to_daqtask-j183 j183.localdomain done | item: /etc/flp.d/qc/stfb_to_daqtask.json | stdout: Success! Deleted key: o2/components/qc/stfb_to_daqtask-j183 j183.localdomain done | item: /etc/flp.d/qc/hmpid_raw_qc.json | stdout: Success! Deleted key: o2/components/qc/hmpid_raw_qc-j183 j183.localdomain done Store machine-dependent qc configuration files to consul... Entry updated: readout_to_daqtask-j183 Configuration imported: qc/ANY/any/readout_to_daqtask-j183 | stderr: Loading coconut/v0.22.2-1 Loading requirement: BASE/1.0 Entry updated: stfb_to_daqtask-j183 Configuration imported: qc/ANY/any/stfb_to_daqtask-j183 | stderr: Loading coconut/v0.22.2-1 Loading requirement: BASE/1.0 Entry updated: hmpid_raw_qc-j183 Configuration imported: qc/ANY/any/hmpid_raw_qc-j183 | stderr: Loading coconut/v0.22.2-1 Loading requirement: BASE/1.0 Entry updated: mft-digits-qc-j183 Configuration imported: qc/ANY/any/mft-digits-qc-j183 | stderr: Loading coconut/v0.22.2-1 Loading requirement: BASE/1.0 Entry updated: mft-raw-qc-j183 Configuration imported: qc/ANY/any/mft-raw-qc-j183 | stderr: Loading coconut/v0.22.2-1 Loading requirement: BASE/1.0 j183.localdomain done Delete old qc common configuration files to consul... j183.localdomain done | item: /etc/flp.d/qc/postprocessing_daqtask.json | stdout: Success! Deleted key: o2/components/qc/postprocessing_daqtask j183.localdomain done Store common qc configuration files to consul... Entry updated: postprocessing_daqtask Configuration imported: qc/ANY/any/postprocessing_daqtask | stderr: Loading coconut/v0.22.2-1 Loading requirement: BASE/1.0 j183.localdomain done Open port range for Consul health checks... j183.localdomain ok | msg: Permanent and Non-Permanent(immediate) operation Ensure telegraf 1.17.3 is present... j183.localdomain ok Detect whether ReadoutCard is installed... j183.localdomain ok | stdout: v0.31.2-1 Set ReadoutCard version... j183.localdomain ok Retrieve InfluxDBv2 token... j183.localdomain skipped Set InfluxDB v2 token variable... j183.localdomain skipped Deploy main configuration file... j183.localdomain done Creates scripts directory... j183.localdomain ok Deploy CRU monitoring script... j183.localdomain skipped Deploy IB monitoring script... j183.localdomain ok Add telegraf user to pda group... j183.localdomain ok Ensure telegraf is enabled... j183.localdomain ok Make sure unix socket has correct permissions... j183.localdomain ok Ensure telegraf is started on boot... j183.localdomain ok Check if consul is up at http://j183:8500... j183.localdomain ok | stdout: o2/ Register output of o2-roc-list-cards to file... j183.localdomain done | stdout: { } Register a o2-roc-list-cards dict for internal use... j183.localdomain ok Push an empty FLP hardware key to the Consul KV store if no cards present... j183.localdomain done | stdout: Success! Data written to: o2/hardware/flps/j183/ Push an empty Detector FLP hardware key to the Consul KV store if no cards present... j183.localdomain done | stdout: Success! Data written to: o2/hardware/detectors/TST/flps/j183/ Push the o2-roc-list-cards json to the Consul KV store... j183.localdomain skipped Push the o2-roc-list-cards json to the Consul KV store for detectors... j183.localdomain skipped Check if consul already populated w/ configs... j183.localdomain done Push CRU default configurations to Consul KV store... j183.localdomain skipped Clean files in /tmp... j183.localdomain done Run o2-roc-config service (via handler)... j183.localdomain skipped Restart telegraf (via handler)... j183.localdomain done Cleanup meta/slaves/latest (via handler)... j183.localdomain done Restart mesos-master (via handler)... j183.localdomain done Restart mesos-slave (via handler)... j183.localdomain done [WARNING]: Could not match supplied host pattern, ignoring: qc - qc on hosts: qc - No hosts found! - Play recap - j183.localdomain : ok=450 changed=113 unreachable=0 failed=0 rescued=0 ignored=2 SSH key is not found. ssh to root@j183.localdomain using the keyboard interactive method. Password: Please reboot all machines for changes to take effect. [flp@j183 ~]$