The 10 Biggest Data Collection Challenges and Frustrations: A Researcher's Guide

Pain Points in Human Data Collection

‍

We recently held discussionsp with researchers from academia and industry across a wide variety of disciplines: nutrition, public health, neurodegeneration, cancer, immunology, microbiome research and more. Many users working in human research and collecting human data were vocal about some of the most important points to them when weighing up the pro's and con's of data collection.

‍

1) Not another "Portal"...(!)

‍

Digital solutions can be a great thing for productivity, but a positive onslaught of entry points to a study's inner workings is not only cumbersome, it's also highly insecure. Usernames, email addresses and passwords are spread across many platforms and humans are always the biggest risk to data breaches, loss and errors. Some researchers even owned up to storing passwords on post-it notes around their workstations because 'there were simply too many logins to remember'. If none of these platforms have stringent password policies (I am of course referring to the "password123" users) and no two-factor authentication, this is surely a recipe for disaster.

So how do we remedy this? With new tools scattered across the web - both paid-for and open-source for the benefits that they uniquely offer - how can we operate in such a way that we need fewer of them? All-in-one solutions with rich integrations within them to existing workflows are essential.

Many organisations will predominantly use Microsoft or Google G-Suite services operationally on a day-to-day basis. Providing integrations through API linkage and single sign-on (SSO) dramatically cuts down on risky access points and gateways for malicious outsiders to find a way in. But what if you are already using a number of services - are they talking to each other? are they sharing information with one another effectively? This very pressing point brings us to our next topic.

2) Nothing we use talks to anything else...

‍

Software often comes with unique IP - whether this is a particular feature, algorithm, mode of productivity or even a protocol. Things are certainly improving with a lot of cross-talk between digital services, but platforms which don't offer this "collaborative glue" for multiple services to interact with one another (e.g. application program interface or APIs) are creating problems and risks to data and security for researchers every single day.

We talked earlier about the hidden costs associated with data monitoring and data audit - and here are more costs that escape our notice. Many researchers admitted having to copy and restructure data tables from one platform to another to work with them, bringing about more risks of accidental data loss, and errors in formatting and manipulation which can all jeapordise the outcomes of the study.

3) Platforms we have tried are too difficult to use

‍

A key component of any software is the user experience. User-friendly headers, fonts, colours, guidance and good visuals make the experience more pleasant and productive. When end users need to use platforms for many hours a day, why make the experience difficult for them?

Cloud-based services are hugely beneficial in this context. By using a software platform which runs on a browser, you're not limited to an installation, a particular operating system and its fonts and colour schemes. Web platforms are designed to work optimally in an isolated container in your favourite web browser. They're also built to scale from device to device, so whether you are using a desktop, laptop, tablet or smartphone, the designer of a web app will have taken these into consideration.  

When platforms are difficult to use, it's crucial that there is a point of contact, whether this is up-to-date documentation or a customer support contact.

4) There is no support, training and no actual humans to talk to...

‍

Many out-of-the-box solutions will come with some sort of manual or documentation. Reading through this can be cumbersome and if you don't have a technology and/or innovation champion in your team, getting set up can be incredibly intimidating. Without IT support, you might need to elect a champion or digital torchbearer in your department who will adopt and lead on the operations and management of your digital systems. Once again, a hidden cost emerges.... An otherwise qualified and trained person committed to another important job role has been fragmented into another critical role if software solutions are unable to provide support.

5) IT teams are required for our platforms of choice

‍

Speaking to many academics using open-source platforms for data collection, they highlighted some concerns surrounding IT resources and support. Some organisations and universities could not get the support of in-house IT to install, host, uphold and maintain data collection services, especially for human clinical trials. In fact, many researchers were surprised that free solutions still needed the provision of IT professionals to operate those solutions - a salary cost they presumed to have been allocated by the host organisation.

Cloud service providers could provide significant benefits in this regard. If cloud platforms self-host and adopt this responsibility from the outset, they will need to be fully transparent with their security protocols and will be required to be compliant with international regulatory frameworks. In addition, they will be required to support and provision backups and encryption of data. If you are adopting more digital solutions, security and data ownership are key.

6) Data, encryption and security concerns

‍

The fear of many is that sensitive data - whether it’s patient health records, or sensitive information pertaining to company IP - is not safe. How do you ensure that any platform or method of working is secure? Here are 10 points that you should consider when you're implementing any digital system - especially for use in clinical, nutritional or public health research.

• Is the login process sufficiently secure?
• Are password policies, 2FA and SSO supported?
• Are login credentials and passwords encrypted?
• Can you prove who had access, what they did, and when?
• Are sensitive data encrypted during transit between devices?
• Are data encrypted at rest on servers?
• Are data backed-up?
• Can you export your data as and when you need to
• Can you get full audit logs  
• Do all of these best practices follow necessary regulatory compliance guidelines?

... Who is Who?

Even if you can provide evidence to show that your methods of working are secure. Do you have sufficient control over users outside of the workplace? In a post-COVID-19 way of working, many groups have switched to off-site data collection - often involving many users and collaborators from different organisations. This method gives researchers, patients or participants in any research the freedom to take part when it suits them and can also greatly improve compliance by reducing their burden. However, shareable links to websites to fill in forms are often not encrypted and not secure. From a regulatory compliance perspective, once a participant has gone off-site and begins submitting data to your study - how can you be sure that it is them? With just a link that anyone could access and no secure login methods, can you truly be sure?

In the newly decentralised way of working, not only are you responsible for ensuring internal operations are secure within your immediate team members, but also that externals such as participants interacting with your systems are also just as secure, reliable and accurate. Many regulatory guidelines including those set by the MHRA/HRA in the UK, EMA in Europe, and the FDA in the USA will require that you can prove with sufficient evidence that users are who you expect them to be and that any data you assume to be submitted by them is indeed the case.

7) Regulatory compliance - what is up to scratch?

‍

The handling of sensitive information comes with the often heavy burden of regulatory compliance. The problems around DIY and makeshift solutions are present in both academic and industrial settings. Most clinical research teams will be very familiar with expectations from regulatory health authorities and good clinical practice (GCP) training is usually a bare minimum for any staff working in this space. Is the software partner aware of these regulatory compliances? Are they a team of developers? scientists? clinical researchers with sufficient knowledge of the industry?

If your organisation is using a self-hosted solution, is it set up to be compliant? Some researchers admit to being unable to use some products because their IT infrastructure is unable to support the level of security needed to comply with regulatory standards.

Since the hay day of the silicon valley explosion, many software as a service (SaaS) products have come to market. While there is certainly developer expertise to make great products, if you are working in clinical, nutritional and digital health research it's essential that your software provider is aware of these regulatory guidelines and requirements. Should a worse-case scenario arise, you will want to know that, first and foremost, participants have been protected as much as possible, and that all protocols were followed.

8) Nothing does everything that we want

‍

Many researchers wanting an all-in-one solution might have quite specific requirements. Their data collection might be text-heavy, or, have very high storage requirements. An example of this would be image or video data collection. Perhaps they need to collect data on cognitive performance and need these tests incorporated into the platform, or, they need to connect to other services such as food diary entry databases.

Good software partners should be able to work with you to implement these solutions and, in some cases, provide some bespoke options. If they are unable to, the API method of linking services is also a must. Closed systems where there are no people to talk to come with risks of flat-out rejection. Consequences of this are, as you might have guessed, multiple platforms being used, more logins and security risks, poorer user experiences, time-wasting and data duplication and many more points we have already discussed.

9) Software can be expensive and there is no budget for this

‍

If you choose to take the software route for your data collection, there will always be a cost - invisible to not. Whether software license costs are covered by your host organisation or a grant, or whether it is a free service with absorbed costs through staff time or department allocation - they are never truly 'free'. There will always be either a monetary or time cost or a combination of the two. In this regard, the best solutions must weigh this up and provide the most added value across these two pivotal points.

Academics especially were often unwilling to use paid for services

Most researchers will have access to a work terminal with unique logins and internet access - but where is the data collected? On servers hosted by IT services? on a shared drive?

Here were some of the major concerns with were raised about the software

Open-source or free solutions

Pro's

• Software and/or code is freely available
• Options for modification and re-distribution under free license agreements
• If set up correctly, can be used indefinitely
• Can be more widely used as they are free and therefore have more of a 'community' feel

Con's

• Often no support or 'humans' at hand to talk to
• End-users are responsible for setup, maintenance, backups and training
• Sometimes requires the re-allocation/commitment of staff members or even new employees to work on the platform indefinitely
• Responsibilities are still mostly yours
• They might not be updated (security or otherwise) as regularly
• If something is 'free' there was a feeling to place less emphasis on responsibilities or accountability across all parties

Paid for or subscription solutions

Pro's

• Updates are usually free or included
• Support is usually costed-in or can be arranged (can be free in some cases)
• Responsibilities are either shared or greatly reduced as part of the service agreement
• Additional services can be added on as needed
• The product is often worked-on and updated very regularly
• Since responsibilities weigh upon the software provider, they can sometimes be more committed to the product
• Efficient pipelines and automations can save staff costs or even reduce the need for additional team members
• More likely to include integrations with other platforms
• Can be worked with to develop bespoke solutions or integrations

Con's

• Software which requires installation requires hardware, network storage space, IT support and maintenance and a lot of internal infrastructure
• Some users or grant funders are unwilling to pay license costs
• Requires some level of training
• Updates and new features might require new training

10) What we use is unreliable or untrustworthy

‍

Many researchers using traditional and hybrid approaches had many concerns about their current set-up. What they had implemented was described as 'not perfect, but did the job closely enough...'.

Traditional (paper, pen, phone call) issues

• Some letters sent out by post were lost or not returned
• Illiteracy and illegibility concerns with written documents
• Many users admit to having poor written skills since using keyboards and devices, reducing legibility
• In recent years, people increasingly don't answer phone calls - especially from unfamiliar numbers
• People didn't want the burden of lots of physical documents

Technology (email, app, web app, portal) issues

• Email systems were not always reliable and many critical messages were not delivered or sent to spam
• Not everyone has a smartphone or the capacity/storage to install apps
• Systems which were not user friendly enough left many users (researchers, participants and more) unwilling to take part
• People don't like to remember multiple passwords and sometimes use the same ones across platforms, increasing security risks

‍

Your thoughts?

If you are working in human data collection and have more to share in this space, feel free to get in touch at hello@trialflare.com