Computation and Analysis
There are major research challenges faced by academics and industry professionals in ‘Big Data’ analytics, especially concerning economic and financial analysis. In order to conduct world-class research, scientists currently require extensive computing expertise in low-level data management and programming to do ‘deep’ data mining and build computer simulation models. The other major research challenge is the access to and handling of sensitive and expensive ‘real-world’ data.
These computational, analytics and data challenges are significant research ‘blockers’ for scientists, policy and industry professionals, which our team are exploring as part of our programme of work.
⎪ Advanced analytics – few mathematical and social scientists have the programming abilities required to fully utilise the computational capabilities of machine learning, computational statistics and statistical physics models. They need configurable libraries which also allow models to be combined.
⎪ Multiple data sources – academic researchers and industry professionals are increasingly seeking to combine multiple historic and real-time data sources, such as financial, economic, social media, and spatial data. Open data sources are also changing this landscape and the ground rules for access.
⎪ Low-level data management – handling multiple data sources and real-time streamed data currently requires researchers to program Hadoop and noSQL servers and also real-time feeds.
⎪ Commercial data – from a research viewpoint some of the most exciting opportunities involve data held by government, financial institutions, data providers, telcos. Currently the only way to access such data is for graduate students and staff to work on the data at the organisation and under strict NDA. Communication protocols to support secure decentralised analytics are required, building upon the experience of two ESRC ‘Big Data’ investments – the Administrative Data Research Centre for England and the Consumer Data Research Centre (both based at UCL).
⎪ User dashboard – a user interface is required that handles all the low-level ‘computing’ technology.