GARR: NOC home working like a boss
It was 16 March 2020 and the Italian lockdown had just been applied at the national level a couple of days before.
At GARR, the Italian NREN, part of the services had already been adapted to remote use therefore the switch to home working was immediately possible for part of the personnel. Still, some services required the presence of staff on site but, in less than a week, we made the transition to full remote working for everyone.
How was it possible to guarantee the functioning of such a complex infrastructure as the one managed by GARR, even in this emergency situation?
The efficiency of the network is checked in real-time by a team that is used to play in advance and prevent accidents, thanks to the experience gained over the years and to the monitoring tools available . We are talking about the Network Operation Centre, better known as NOC, considered the core of GARR, which in a very short time could successfully address the situation by reconfiguring its processes to be suitable for remote working. We interviewed Alessandro Inzerilli, head of NOC and Operations groups, who manages GARR network operations.
GARR-NOC was the first among European research networks to organise its work remotely in such a short time, how did you manage that?
I can definitely say that communication and collaboration are deeply rooted in the way we use to work. In normal circumstances the NOC team works at GARR headquartes in Rome and manages a network with over 100 points of presence (PoP) and more than 1.000 connected sites, with the collaboration of local network administrators and the support of our suppliers. What we mainly did was to adapt to the new situation the procedures and working tools that were already in use.
These tools are the GARR Integrated Networking Suite (GINS) and other applications for monitoring and identifying problems on the network, as well as the VPN software to remotely manage the IP and transmission network. In addition, I am referring to all those tools facilitating teamwork, such as video conferencing app (OpenMeet and WebMeetings) for team meetings, chat tools, Onlyoffice for sharing documents and Jira for managing work flows.
With reference to Jira, I created three projects concerning network problems, daily operations and the management of network upgrades: each project is then structured into sub-activities assigned and carried out according to an identified workflow. In this way it is possible to easily keep track of the status of activities, deadlines and updates with no need for further communications. It is therefore a tool to manage activities and workflow which can be very useful in an agile work environment. The last thing we implemented was the extension of VoIP (Voice Over IP) capabilities to all staff, using the Asterisk platform. This system, developed with the support of the colleagues of the System group, helped us manage the incoming calls directly from home.
Which critical issues are you facing? Did your daily operations change?
Sometimes physical access to PoPs and user sites and can be difficult, as most of them are hosted by the organisations connected to GARR. Whereas it is easier to get access to our PoPs hosted by TLC operators. At the moment, we are giving priority to the availability and stability of the network to ensure the same level of service to the whole user community. We do not send our technicians to the sites when minor accidents occur, we are postponing hardware installations and non-critical network changes and we are giving priority to more urgent upgrades.
Did you notice any change in support requests?
The huge numbers of students accessing live streaming classes and other learning materials from home generated a hike in the upload traffic on GARR backbone of 60% compared to the annual average. Just to give some examples, the Polytechnic of Turin recorded an increase of the traffic in upload of 183%, the University of Milan an increase of 185%, the University of Siena an increase of 231%, the University of Trieste an increase of 185%. We also experienced an increase in demand of video conferencing systems for e-learning and teamworking reasons. GARR was already providing video conferencing systems such as Vconf and Webmeetings but we decided to expand the resources for the community by participating in a nation-wide initiative offering free and open source solutions OpenMeet and EduMeet, for schools and other users . In particular, we made the servers available on the GARR cloud with the support of our Computing and Distributed Storage Department. In relation to the videoconference system, we immediately recorded an increase in numbers: +800% of videoconferences on Webmeetings compared to the average of the previous 5 months and more than 2000 conferences per day on iorestoacasa.work with over 20.000 video meetings carried out only in the first two weeks.
We are also implementing some changes to improve the connectivity service for the Italian Research Hospitals (IRCCS) which are all connected to GARR. These hospitals are specialised institutions dedicated to research in the biomedical field and are currently increasing their research and experimentation activities for the COVID-19 response, such as new drugs and vaccines. For this reason they have an increasing demand for bandwidth, as demonstrated by the traffic statistics of many of them.
How do you manage the external communication with the user community?
We inform our user community using different channels. We communicate through the institutional GARR website, the NOC website and the CERT website (for the security part), we also send updates through our mailing list and we communicate on a daily basis through our social media channels such as LinkedIn, Instagram, Facebook and Twitter.