Introduction
The PSDI Data Conversion Service is a one-stop shop for scientific file format conversion. The service enables users to seamlessly convert between different file formats and view information about the quality of the proposed conversion.
Challenges and Impact
A key challenge in the physical sciences is a lack of FAIR (Findable, Accessible, Interoperable, Re-usable) data. PSDI recognises that there is a strong need within the physical sciences community for a simple and effective way to convert between data types and file formats for researcher collaboration and to facilitate data exchange between different services. Research workflows often require the use of different data formats between different processes and software in research pipelines making interoperability a significant challenge. Many instruments and software vendors provide data in bespoke formats making it complex and time-consuming to convert between different formats. Conversion is often required again to get data into a standard format to facilitate collaboration, publication, and preservation. The accessibility and usability of tools that already exist for data conversion is variable, and it is often not easily apparent what the quality of the conversion will be without significant manual testing.
The PSDI Data Conversion Service aims to address these interoperability issues by providing an easy mechanism for researchers to convert between different scientific data formats, thus removing barriers to interoperability and facilitating easier data sharing and collaboration. The service addresses several key challenges with managing data in the physical sciences:
- difficulty in reusing data due to incompatible formats or unsupported software
- the effort needed to determine which data conversions are possible
- challenges involved in installing and operating conversion software
- the time required to understand which conversion tools provide the best results for a selected conversion
PSDI Data Conversion Service
The PSDI Data Conversion Service offers a range of benefits tailored to meet the needs of physical scientists, making it a valuable tool for data management and transformation. It aggregates comprehensive information about scientific file formats from multiple sources, providing an extensive array of potential conversions along with insights into the options available across various platforms. Quality metrics accompany the conversion information, helping users assess the effectiveness and reliability of different services. The service simplifies the data conversion process, eliminating the challenges of locating, installing, and operating existing tools. Users can explore the full range of conversion possibilities with ease, enabling them to make informed decisions. Additionally, the service is flexible, offering both online operation for quick access and a local option for maintaining data within personal infrastructures or incorporating conversions into research pipelines. Feedback from users is actively encouraged to enhance the service, ensuring it evolves in response to the needs of the scientific community.
PSDI Data Conversion Service Highlights
PSDI provides a number of different ways to access and use the Data Conversion Service to meet the needs of different users:
- Online PSDI Data Conversion Service: This service provides a user-friendly, web-based solution for exploring the conversion options of different scientific file formats and performing occasional file format conversions without the need for technical expertise. It provides quality measures where possible for available converters and directs users to external services if a conversion cannot be performed.
- PSDI Data Conversion Container Image: This service provides the same features as the hosted web application while guaranteeing full data privacy. No data is transmitted to PSDI, and no user metrics are recorded. It is ideal for users with technical expertise who value data privacy and prefer to operate the conversion service on their own infrastructure. (This option is targeted to be available by the end of April 2025).
- PSDI Data Conversion Command Line Application: This service offers all the conversion features of the web application, with the added benefit of running locally, eliminating file size limitations. It is particularly suited for technically skilled users who prefer command line tools and need to process large datasets without restrictions.
- PSDI Data Conversion Python Library: This library provides the same conversion capabilities as the web application and command line tools, with no restrictions on file size. It is ideal for automating conversions within tailored data workflows, making it a great choice for researchers, data scientists, and developers looking to integrate file format conversion into Python-based data processing pipelines.
You can access all the different versions of the Data Conversion Service that we offer from the PSDI Data Conversion Resource Theme entry in our What We Provide page. You can go straight to the online version using the PSDI Data Conversion Service link.