Organize

Organize

File & Folder Organization

File & Folder Organization

Organizing your research files with a consistent system will make your data easier to understand, share, and archive. When you revisit your data files over time, it will take less effort to find what you’re looking for. There is no one right way to organize your files, but the key to a good system is that it should be descriptive and standardized, and to implement it consistently. When you’ve come up with a file organization and naming system, describe it in your research project’s readme file.

File Naming Conventions

Important pieces of information to include in filenames include things like:

  • Short description of file contents
  • Date created as YYYYMMDD or YYYY_MM_DD
  • Project name or acronym
  • Initials of researcher (if working on a collaborative file)
  • Version number
  • Other important metadata such as location or

Other important practices in file naming are:

  • Avoid special characters such as & , * % # * ( ) ! @$ ^ ~ ‘ { } [ ] ? < > –
  • Try to keep names short, no more than 25-30 characters
  • Separate parts of filename using underscore _ or by using CamelCase where every word is capitalized
  • If you are writing your own code or working with code, avoid using spaces in file names

Example:

LakeMercury_TestData1_2020_10_01_TM_v3.csv

20201001LakeMercuryTestData1TMv3.csv

  • LakeMercury = Project Name
  • TestData1 = Description of file contents
  • 2020_10_01 or 20201001 = data collection date
  • TM = Tracy MacDern
  • v3 = Version 3

Data Documentation

Data Documentation

The palest ink is clearer than the fondest memory

-Chinese proverb

Raw data isn’t easy to understand or re-use. To make it easier to understand, include descriptive metadata and follow the FAIR principles, which state that data should be:

  • Findable
  • Accessible
  • Interoperable
  • Reusable

The easiest way to build documentation for your data is to create documentation files like:

  • ​​​​​A readme file: a document that describes the files and organization of your data. This helps you remember what you've done years after you've done it, and helps others interpret the files and folders inside your dataset if you decide to share/publish the data. They should be simple text documents (.txt, .md, or .pdf). Data repositories are increasingly starting to ask researchers to provide readme files, because they dramatically increase the legibility of datasets. They typically include:
    • Basic project metadata: contact information and links to associated publications and data sources
    • A description of the project's folder hierarchy and file naming scheme
    • Description of the contents of important data files
    • For more information on building readme files, see the Harvard Longwood Readme file guide and this post by data professional and author Kristin Briney called README.TXT
  • Data dictionaries are documents directly describing data elements and variables. They typically accompany tabular data. Data dictionaries have several benefits:
    • Keeping things consistent across a project with multiple researchers. The dictionary can define data names, labels, units, and can define data constraints by providing an acceptable range of values.
    • Providing “human-readable” details to support interoperability and re-use by helping others understand the datasets
  • Codebooks are similar to data dictionaries but for survey or statistical data. In the case of survey data they can include the layout and structure of a survey, and define the codes used for questions and answers within the survey. They can be automatically generated by software like REDCap or SPSS or manually created.

 

 

Metadata

metadata text

    The first step towards making a dataset FAIR is creating good metadata. Metadata is "data about data", and includes simple elements such as a dataset title, description, author list, date of creation, language, and a unique identifier/link such as a DOI. But it can also include much more discipline specific information such as measurement abbreviations, protocol information, experiment parameters, geographical information, and more. Many research communities use metadata standards, which can be domain-specific, or cross-disciplinary. Some examples include:

    • Dublin Core - a basic, discipline agnostic standard, widely used and easy to understand. 
    • mmCIF - Macromolecular Crystallographic Information Framework, a metadata standard used for macromolecular structural data.
    • EML (Ecological Metadata Language) - specific for ecology disciplines
    • BIDS (Brain Imaging Data Structure) - a standard for neuroimaging and behavioral data

    The UK Digital Curation Centre (DCC) hosts a list of metadata standards that can help you find the most appropriate standard for your data. The Research Data Alliance maintains an open directory of metadata standards.

    Research Project Management

    Research Project Management

    There are an increasing number of software platforms designed to enhance research collaboration.

    Many document creation tools like Google Docs and Microsoft Office now offer real time collaborative document creation and editing. These tools can help avoid the confusion that comes with multiple versions and copies of documents being passed back and forth by email.

    Reference Management tools such as Zotero, Mendeley, or Endnote (see the Library's Comparison Page for a comparison of these programs) can also support group work through shared citation libraries.

    The Open Science Framework (OSF) is a free open platform for research collaboration. OSF provides a place to manage files, data, code, and protocols in one centralized location. Research can be organized into projects and components. Access can be provided to collaborators and to the public (if desired). Files on OSF have automated version control, and can be registered to keep snapshots of different phases of the work, such as pre-registering a hypothesis or analysis plan. Tools such as citation managers, data storage providers, or repositories can be easily linked to OSF on a project by project basis.