Data storage and project organization

Where is the data on the cluster?
On the platform, all datasets are typically stored and organized under a project with a particular name. This is a way to manage security and access rights for individual users. When connected to the cluster via ThinLinc or from a “DCC Machine”, the data of active projects is found at

  • /dtu/3d-imaging-center/projects/

The project folder has a large number of different projects, but a user will only have access to the projects that he/she is involved in.

💡 Symbolic links: As the amount of projects grow, the main folder tends to become quite overwhelming for browsing and navigating. Consider using symbolic links to the folders that you typically access (see below).

1. Project organization:

Each project is organized with the following structure:

  • analysis/
  • raw_data_3DIM/
  • raw_data_extern/

The raw_data_3DIM folder should contain folders with the ‘raw’ data (reconstructed volumes and/or projection data or whatever type is relevant) that was obtained at the 3D Imaging Center.
NB: General project members have read and execute access (+rx) to this folder, to prevent accidental modifications of the ‘raw’ data.

The raw_data_extern folder will host datasets collected elsewhere, and can be managed by anyone in a particular project.

NB: All members of the project will as default have full access (+rwx) to this folder, so be mindful of what you do when accessing data in this folder!

The analysis folder is where the data processing should take place. The folder can host project relevant software environments and functions, processed versions of the datasets, figures, reports and what-not.

2. Symbolic links to projects

The project naming might not always be very logical, and even hard to remember – in particular when not accessing a project regularly. In that case, it can be convenient to create symbolic links on the ThinLinc desktop.

Manually:

  1. Open the File Manager
  2. Navigate to /dtu/3d-imaging-center/projects
  3. Right-click on desired project folder and choose “Send To Desktop (Create Link)

Via terminal:

  • Launch a terminal on the ThinLinc desktop.
  • e.g. right-click on the ThinLinc desktop background, and select “Open Terminal Here“.
  • Create a folder named <FOLD_NAME> which links to a desired folder <FOLD_LINK> by calling
  • ln -s <FOLD_LINK> <FOLD_NAME>
  • e.g. ln -s /dtu/3d-imaging-center/ 3DIM