Case Status
Log In

Wiki

Options

 
Tool Tips
  • RSS Feed

Last modified on 12-05-2015 15:03 by User.

Tool Tips

 


Retrieving a large file using wget

The following parameters to wget will make wget continually try to resume a file, eventhough the connection is broken temporarily.

$ wget -c --timeout=120 --waitretry=60 --tries=10000 --retry-connrefused ftp://url.somewhere/important.file

Remember, always to retrieve large files in a "screen", in case the connection to the terminal is interrupted.



The screen command

Use screen to run multiple consoles in the background.
It has the huge advantage, that your session will continue even though you shut down your laptop or is disconnected from GenomeDK.
Just type screen to start.

$ screen

			

To leave and put the console in the background, type Ctrl-A + D
To reopen the console type at a later time

$ screen -r

			

Screen has lots of other nice features. See man screen


Increasing the BASH history

something@something:~$ export HISTSIZE=100000 (default size is 1000)

To fetch commands from your history, use Ctrl-R

something@something:~$
(reverse-i-search)`vn': vncviewer -via login.genome.au.dk fe1:2

 

Running RStudio on the cluster

RStudio is available on the cluster as a desktop application, which can be run on both compute nodes and the frontend node. Bare in mind, the frontend node must not be used for computation or analysis. The GUI of RStudio is shown using X-forwarding, thus you must have an Xserver installed locally on your machine.

Xserver software:

  • Linux - preinstalled
  • MacOSX - XQuartz - http://xquartz.macosforge.org
  • Windows - MobaXterm - http://mobaxterm.mobatek.net  (SSH client bundled with an Xserver)
  • Windows - Putty + Xming - http://sourceforge.net/projects/xming/  (remember to start Xming before connecting with X-forwarding)

To forward the GUI of applications, you must tell SSH (or putty) to enable X-forwarding.

SSH: Use the '-X' parameter to enable X-forwarding

user@localmachine:~$ ssh -X user@login.genome.au.dk 

Warning: untrusted X11 forwarding setup failed: xauth key data not generated

Last login: Tue May 12 13:39:15 2015 from 130.225.0.251

     ----------------------------------------------------------------------

     |                                                                    |

     |                         Welcome to GenomeDK                        |

     |                        ~~~~~~~~~~~~~~~~~~~~                        |

     |                                                                    |

     ----------------------------------------------------------------------

 

user@fe1:~$

 

Putty:

When logged into a terminal, you can now load the RStudio into your environment and start it:


				

user@fe1:~$ source /com/extra/RStudio/LATEST/load.sh

user@fe1:~$ rstudio

 

To run an analysis or computations in RStudio you will need to run RStudio in an interactive job on a compute node.


					

# Request 1 core, 4gb memory for 10 hours.
user@fe1:~$ srun --mem=4g -c 1 --time=10:0:0 --pty bash

srun: job 3597082 queued and waiting for resources

srun: job 3597082 has been allocated resources

[user@s03n11 ~]$ rstudio

 

RStudio is automatically terminated if it allocates more than the reserved 4gb, the 10 hours expires or the connection is lost. So, remember to save your work!

To check your job usage. Run jobinfo in another terminal:


					

[user@fe1 ~]$ jobinfo 3597082

Name                : bash

User                : user

Partition           : normal

Nodes               : s03n11

Cores               : 1

State               : ,RUNNING

Submit              : 2015-05-12T13:53:21

Start               : 2015-05-12T13:53:37

End                 : --

Reserved walltime   : 10:00:00

Used walltime       : 00:04:41

Used CPU time       : --

% User (Computation): --

% System (I/O)      : --

Mem reserved        : 4G/node

Max Mem used        : 118.92M (s03n11)

Max Disk Write      : 40.00M (s03n11)

Max Disk Read       : 18.00M (s03n11)