The coronavirus pandemic is forcing data professionals to work their fingers to the bone, with 97% of data teams either at full capacity or over-capacity due to the increased workload brought on by the virus, leaving little room for them to work on new projects.
That is the stark conclusion of a new study by Ascend.io, which quizzed over 300 data scientists, data engineers, and enterprise architects over current practices, productivity bottlenecks, and their perspectives on automation.
It reveals that, since the onset of Covid-19, nearly four out of five (78%) data professionals have been ordered to take on responsibilities outside of their core job function, with just 3% saying they have spare capacity to take on new tasks.
Moreover, senior data executives are nearly twice as likely to indicate their teams are overloaded than team leads and individual contributors themselves, signaling a significant backlog and increasing demand by the business.
When asked which team was the most backlogged, enterprise architects led the way, followed by data engineers. However, respondents were nearly four times more likely to identify their own team as the most backlogged over others. Despite pointing to their own team as the backlog, the research found compelling patterns emerging around the need for data engineering resources to complete their work.
Slow iteration cycles in data teams are also leading to significant delays, hindering teams’ ability to meet the data needs of the business. The delays vary by role, the study shows, with data scientists impacted by having to ask others for access to data or systems (48%), and data engineers are being held back by maintenance of existing and legacy systems (54%).
Ascend.io chief executive and founder Sean Knapp said: “Organisations are quickly discovering that data engineers are essential to unlocking the value of data and to removing bottlenecks across the entire data team.
“LinkedIn’s 2020 Emerging Jobs Report found that data engineering has surged onto the scene, quickly becoming one of the top-ten jobs experiencing tremendous growth. At present, there are simply not enough data engineers to meet the demand.”
Data professionals are increasingly turning to automated solutions to solve their team’s current challenges. Some 62% of respondents plan to implement automation technology to increase their team’s bandwidth, followed by buying new products or tools (47%), replatforming and retiring legacy technologies (41%), and hiring more staff (41%).
The majority (73%) of respondents view automation as an opportunity to advance their career.
The survey also indicates that across the spectrum of start-ups to large enterprises, data teams are increasingly looking to low- and no-code solutions to accelerate innovation and productivity. More data professionals are seeing the value of these tools, with 80% of respondents already using or considering them to support their team.
Knapp concluded: “The research demonstrates that across the board, teams are being asked to do more with less. Low-code solutions are great at simplifying what used to be incredibly complex tasks. They also tend to encourage better modularity and system design, which keeps things simpler and more maintainable.”
Related stories
Calling all misfits and weirdos: No 10 builds data unit
Data professionals confident of avoiding Covid jobs cull
Crisis? What crisis? Analytics market to top $512bn
We’ll need analytics skills to survive, UK chiefs predict
Data science is thriving – do we know what’s going on?
Gender parity will take more than just one day to fix
Skills crisis: ‘No better time to be a woman in data’
