One of the features that sets the SAIL Databank apart is the ability for researchers to be truly flexible in where they access their anonymised data. Through processes and technological developments crafted over the last 10 years, SAIL has perfected access to secure research without compromise to security, data quality or user experience.
But what happens when the technical team behind SAIL’s success are all themselves working remotely and when the data coming into SAIL now needs to happen on a daily rather than monthly or quarterly basis?
Covid-19 brought its own set of challenges to the technical team behind SAIL and SeRP. The very technologies that so many researchers relied upon to carry out their work wherever they may be was now facing its most significant period of pressure. In addition to ensuring that the hundreds of researchers already accessing SAIL’s secure gateway could continue to do so without any drop in service, the SAIL team were faced with an increased demand for new urgent access from researchers in Wales and across the UK, whilst also processing significantly increased data refreshes to ensure that the information at the heart of the One Wales response was as timely and accurate as possible.
Simon Thompson is the Chief Technical Officer for SeRP – the infrastructure that compartmentalises data for projects to access – and has been at the centre of the team effort to fulfil the One Wales research effort…
“Thanks to the efforts of everyone involved, the scale, scope and cadence of data flows into SAIL has increased with health and administrative data sources now benefitting from daily, weekly and monthly refreshes.”
“The sheer volume of daily dataset refreshes has been a challenge for us, but thankfully it’s something that we’ve really embraced and built from. We’ve moved from receiving some datasets twice-yearly to now receiving them daily and with it the cadence of refreshes has been significant. Because of this, we’ve had to look at new technical solutions to accommodate this.”
Working closely with SAIL as its Trusted Third Party (TTP), NHS Wales Informatics Service (NWIS) has been central to the One Wales effort, by provisioning and managing an unprecedented flow into SAIL of new data necessary to aid Covid analysis.
The NWIS role is twofold; as both the organisation responsible for the split file process that ensures anonymisation of data prior to it reaching SAIL and also as a data provider themselves.
Gareth John is Information Manager at NWIS,
“Covid has brought unprecedented challenges to the NHS in Wales and NWIS have been at the heart of the Welsh response to the pandemic. Without good quality, timely and accessible data, the Welsh response to the pandemic would have been extremely challenging.”
“Due to our existing strong relationships with the SAIL Databank, Welsh Government and many others involved in the One Wales response, we were able to react efficiently to the increasing requests for timely and of the moment data to inform research and national analysis.”
Speaking about the lessons learnt, Simon Thompson,
“Covid has taught us about the organisational aspects of ingesting data. If you do it twice a year then it’s manageable but the recent increased volume has meant that there were significant organisational and project management requirements of the team, to ensure that not only did we continue to deliver as we always had with the same quality, but on a much tighter timescale and with increased urgency.”
“But we’ve viewed this period as something very positive for how we work, what we have built and how we will plan for the future.”
“We now have solutions for users outside of government to transfer data to us even more securely. These solutions also support automatic importing into SeRP and enable the sending organisations to automate the sending of data to SeRP, reducing the likelihood of human errors.”
“This period – working remotely while delivering a service to ensure our One Wales work is delivered alongside all of our existing users’ requirements – has been a good thing for the team all in all. It has pushed our technology and our processes and now we are now redesigning our own technology based upon the lessons we’ve learnt in Wales – this will then be deployed to our SeRP deployments in Canada and Australia for them to deal with their own Covid-19 research challenges.”
The One Wales team will continue to work together to identify gaps in knowledge and streamline efforts to deliver vital intelligence to help policymakers understand and plan around the issue of Covid-19 in Wales and across the UK.
By Cathrine Richards
You must be logged in to post a comment.