A. Download data to your local or attached hard drive


Steps 2 to 4 below must only be done only once, to install the DDSclient on your computer.

If the DDSclient is installed, steps 1 and 5 can be done directly for any project you need to download.


1. Check that you have permission to download project

Click on this link to log into Duke Data Services.

Click log in and you will be prompted for your NetID and Password.

The first time you log in, it will ask you to authorize Duke Data Service. Click Authorize.





You will be directed to your dashboard.

If you are the person who placed the order with the Sequencing Core and received a data delivery email from us, your project should appear on your dashboard.

If you do not see the NGS project listed on your dashboard, it means you are not the owner of that data and need to request access. The owner, i.e. the person who placed the order, can follow the instructions under heading 1.1 to grant you access.

1.1. Grant a collaborator access to your NGS data

If you are the owner of the data and need to grant other investigators download permissions, Click on this link to log into Duke Data Services. Find the project to share on your dashboard.

Click on the on the icon in the upper-right corner of your project. This will open a drop-down menu.

Select 'Add Project Members'.

This opens another pop-up window.

You can 'Add A New Member' by registering the name of the person you want to share the data with.

DDS is linked to Duke's directory. Begin typing the name of a collaborator and select it from the drop down menu. Data on DDS can only be shared with people who have NetIDs (i.e they are in the Duke directory).

You must assign the new team member a project role. Make sure to select either the 'Project Administrator' or 'File Downloader' roles to give download privileges.

Please note that the 'Project Administrator' role will grant that person permission to delete files. 'File Downloader' will be able to download files, but not modify or delete them on DDS.



2. Check that your computer meets all requirements for DDSclient

DDS client tools, which you will need to download your data, are written in Python scripting language. In order to run those scripts, you will first need to make sure Python is installed on your machine.

2.1. Checking Python Installation

Python is probably already installed on your system. Most Apple computers indeed have Python installed by default. To check which version is installed, open a terminal window by going to Applications>Utilities and click on Terminal. You can also press command-spacebar to open spotlight search, type terminal, and then press Enter.

Find out which version of Python is installed by issuing the command python --version

This will output the version of Python installed on your computer.



You need Python 2.7.9 or above (for Python 2) or 3.4.1 or above (for Python 3) for DDSClient tools to work.

If your version of python is too old or you do not have Python (it will state that Python was not found), you will need to install the newest version before moving to the next steps. An easy way to install Python is to download it through the Anaconda suite. Alternatively, contact your IT department or follow Python documentation here.

2.2. check that pip is installed by typing

pip --version

This will return the pip version that is installed. pip is most likely installed on your system but if not, you will want to install it by following the instructions here

3. Installing the Duke DS Client

Now that your system meets all the requirements for DDSClient to work, we can install the DDDclient tools using pip and start downloading (well, almost).

To install the DDSClient, simply run the following command in your Terminal window:

pip install dukedsclient


For DDSClient tools to work, it will need three Python packages: PyYAML, ssl and requests. The ssl Python module is usually installed by default but the other two are most likely not present in your system. However, by installing DukeDSClient using pip, any missing python module will be installed automatically.

4. Config file setup

We are almost done!

DukeDSClient requires a config file containing an agent_key and a user_key.

To set it up follow the instructions below. Windows users, you can refer to this page

1. Go to the Duke [Data Service Portal](https://dataservice.duke.edu/#/login)

2. Click login and login with your NetID

3. Click top left menu button







4. Click **Software Agents**












5. Click **ADD NEW AGENT** button











6. Fill in data for a new agent and click **SUBMIT**.













Pick a descriptive name for your software agent, as it will be associated with the projects you upload.



7. Click **CREDENTIALS** on the software agent you just created.








8. Click **COPY CREDENTIALS TO CLIPBOARD**.














You now have your user and agent key in your clipboard.

9. Go to your terminal window and copy your clipboard to a new file

The new file needs to be called .ddsclient (do not add .txt or any other extension) and needs to be located in your home directory.

To do this, type the following commands into your Terminal window (type what is in bold):

cd ~

This will bring you to your home directory.

nano .ddsclient

This will fire up a file editor within the Terminal window and create a new empty file named .ddsclient for you to paste your clipboard in.

On your keyboard, click Command + V to copy your clipboard into the terminal window. Close the newly created file by clicking Control + X on your keyborad.

Click the "Y" key on your keyboard to save the changes and exit the unix file editor.

You should now be able to use ddsclient!




5. Download your sequencing project

In your terminal window type the following commands (only type what is in bold)

ddsclient list

This will list all the projects you have access to. In our example, only one project is listed. The project name is Alvarez_9000_200331A5






To donwload project Alvarez_9000_200331A5 all you now need to do is use this command

ddsclient download -p Alvarez_9000_200331A5 ./Desktop

It will download the entire content of your project into the specified folder, in this case the Desktop. Currently it requires the directory be empty or not exist. It will create Folder (named like the project) if it doesn't exist.