Specifying and Optimising Data Wrangling Tasks

Data wrangling is "the process of cleaning, structuring and enriching raw data into a desired format for better decision making in less time" [2].

To clean the data prior to analytical tasks, a wide variety of data quality techniques and tools are used [1]. There is also a trade-off between flexibility, performance and usability of data quality techniques and tools [1]. Highly flexible tools tend to overburden the end user with the need to complex application programming interfaces towards expressing quality-aware manipulations over the data. The balance lies somewhere in a spectrum between highly flexible and extensible solutions and less flexible but efficient and user-friendly frameworks. In practice, a combination of complementary tools and techniques may be needed in a data quality management project.

This PhD project aims to investigate popular techniques and tools used by data scientists to conduct data wrangling tasks prior to big data analytics and develop domain specific methods and languages