Archive

Archive

Archiving Data

Services

Good data deserves to be remembered. The best way to keep data from being lost is to make sure it is properly archived. Many important data have been lost over time as storage devices become corrupted and fail, are lost, or are destroyed. 

Research data repositories

Wherever possible, the best way to archive important research data is to upload it to a reputable data repository. When data is in your custody as a researcher, you often have to pay for storage and maintain appropriate regular backups to ensure files are not corrupted. Depositing in a research data repository means the repository curators take care of your dataset...so you don't have to! If you're depositing data, you can also easily make it openly available or share it with restrictions. Even if your dataset is too sensitive to be published openly, it can often be de-identified and deposited under restricted access to archive it. 

Careful long-term storage

However, sometimes data is so sensitive it can't be shared publicly ever, even in a de-identified format. Even if you're not able to share or publish data openly, it's important to make sure data is carefully archived once a project is complete. You don't want to have a dusty hard drive somewhere full of disorganized files that is confusing when you open it up again. Hard drives and external storage devices also have lifetimes and will eventually fail and aren't suitable for long term archival. Our Research Data Storage Finder Tool has an overview of storage possibilities.

Before archiving data for long term storage, there are some preparations to make to your files:

  1. Ensure that the data is stored in sustainable, open file formats. Open file formats help ensure access to your data over the long term as proprietary software can disappear when companies go out of business. 
  2. Clean up your files. Delete any files that are no longer relevant (provided you don't need to keep them for ethics purposes) and organize your files into understandable file names and folders to create an archive-ready package.
  3. Data should be well documented and easily understandable. Research data should be packaged with metadata, so-called ‘data about data’. These files help your future self unpack and understand your files and how they were created. Data documentation is commonly included in readme files, codebooks, or data dictionaries. Metadata should include information about:
    • File title, file format, language, creator, and date
    • Data variable descriptions, including data type, allowable values, and calculations used (if applicable)

 

Sustainable File Formats

File Formats

Sustainable file formats typically have the following attributes:

  • Standardized
  • Well documented
  • In common usage
  • Unencrypted
  • Uncompressed

Examples of sustainable formats include:

  • Text: PDF/A, RTF, TXT, XML, LaTeX, md
  • E-Books: EPUB, PDF/A
  • Audio: FLAC
  • Image: TIFF, SVG, PNG
  • Medical Images: DICOM
  • Spreadsheet: CSV, TAB
  • Video: MP4

For more information check out:

Data Curation Primers for a wide range of file formats from the Data Curation Network (DCN)

  • Data curation primers are peer-reviewed, living documents that detail a specific subject, disciplinary area or curation task and that can be used as a reference to curate research data. 

DataverseNO preferred file formats 

Data Retention

Data Retention

When you're considering archiving, you might ask yourself, "How long should I keep my data?" The short answer: As long as possible! But it depends on your data--a balance of PI preference, funder requirements, and ethics boards determine data retention lengths. 

Here are some details to consider:

  • What is useful to keep? Can data be used in other projects to support future research?
  • You will need to find a balance between the risks and benefits of retaining or deleting their data, paying special attention to how identifiable or risky the data are. To learn more about the risk level of your data see the Research Data Management Matrix. Can you keep only a de-identified or anonymous version of the dataset?
  • Make sure you have a plan to manage your data securely on an ongoing basis. Ask yourself: what is my data management plan if I move on from this institution, this field, or this career?
  • Note that some funders, scholarly journals, organizations, and scholarly societies have guidelines about data retention for publication. Raw data (primary data collected by the author and secondary data not otherwise available) and analysis data supporting publications should be kept to enable replication in case a concern is flagged by the publishing journal in the future.
  • Pay attention to where you are in the world. Canada’s Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS-2) and the UK’s Data Protection Act do “not set out any specific minimum or maximum periods for retaining personal data”. However, the U.S. Office for Human Research Protections requires research records such as consent forms or transcribed anonymized interviews to be held for at least three years after completion of the research.
  • If your data is subject to a data sharing or transfer agreement make sure to check the agreement - it may impose stricter requirements.
  • Researchers are encouraged to share or archive their data by uploading it to a data repository such as McMaster Dataverse, McMaster’s institutional data repository.

Health sciences research and Clinical Trials

The Hamilton Integrated Research Ethics Board (HiREB) notes that for clinical trials with Health Canada oversight, researchers must retain the study information for 25 years. For all other data, identifiable data should be destroyed as soon as possible, de-identified data should be retained indefinitely. 

The Tri-Agency Open Access Policy on Publications notes that recipients of CIHR grants are required to retain original data sets for a minimum of five years after the end of the grant (or longer if other policies apply). This applies to all data, whether published or not. 

In the US, the National Institutes of Health (NIH) requests 3 years following grant closeout.

Research Records

Research Records

Research records are the documentation surrounding the research process, and can include consent forms, research proposals, grant applications, and other administrative or financial documents. These documents may have specific retention periods prescribed by ethics agency policy, law or granting agencies.

Paper records

If you're working on a longitudinal study, there's a good chance you have paper records like consent forms that you may wonder what to do with in the long-term. If you have made good and full electronic copies and are prepared to retain those securely for the retention period identified, you may be able to destroy original paper copies. This will be contingent on your original ethics application and what withdrawal procedure you offered to participants. In general, it’s a good practice to delete identifiers when you no longer need them.

McMaster's policy on records retention states that the Minimum Legal Retention Period for Clinical Trials is a minimum of 15 years after the conclusion of the trial. If your study is regulated by Health Canada, their documentation on records management outlines on “Transfer of records to secondary medium” that “The transfer of essential records from their original medium to a secondary one may be acceptable if the conditions described in this section are fulfilled.” 

Even if your study isn't regulated by Health Canada, their documentation can be helpful as a guide. They outline that, “The destruction of original paper records following their transfer to a secondary medium may be acceptable with the principles described in this section in place.”

The transfer process should be validated and documented in appropriate procedures, and should ensure that: 

  • measures are in place to verify that the transfer is accurate and done by appropriately trained individuals (e.g. attestation or certification of copies by a person not involved in the transfer)
  • corrections to the original data can be clearly captured in the secondary medium
  • process follows existing standards when possible (i.e. Canadian General Standard Board)
  • the secondary medium allows the successful retrieval and use of the records for the entire (…) retention period

Health Canada also provides some guidance on what to include in electronic records: “Features should include the following, but not be limited to:

  • design to ensure the tracing of any alterations and updates, if permitted, such as source, date and content (i.e. audit trail)
  • back-ups at regular intervals
  • security measures in place and documented to protect against data corruption, whether through accidental deletion, equipment failures, material deterioration, or a variety of other hardware and software problems
  • controlled access to appropriate individuals (e.g. through use of passwords)
  • plan in place for future accessibility (in light of changes over time in technology, personnel, or third-party contractors)
  • location of records that permits immediate access to records for inspection”

Before deleting any study keys or and consent forms, you should make sure you carefully review your ethics forms to see if you discussed when data will be de-identified and if identifiable data will be treated separately from identifiable data. 

Also check for language around whether participants can withdraw from the study. If you stated that participants could withdraw at any time you will need to keep the consent forms and study key for the duration of the retention period to enable that practice. In general, deleting identifiers and de-identifying data is good practice to reduce the risk of a data breach and PI or PHI exposure. Remember if you are looking to modify your research practices for an ethics approved study you should get in touch with your ethics board as you may need to file an amendment.