Difference between revisions of "UNIX recipes"

 
(4 intermediate revisions by the same user not shown)
Line 15: Line 15:
 
  sudo mke2fs -c /dev/sda1
 
  sudo mke2fs -c /dev/sda1
  
 +
= R tricks =
 +
 +
R locale setting for Mac OS X
 +
 +
To enforce US-english setting regardless of the system setting :
 +
system("defaults write org.R-project.R force.LANG en_US.UTF-8")
 +
 +
Note that you must always use .UTF-8 version of the locale, otherwise R.app will not work properly..
 +
 +
= System monitoring =
 +
 +
list processes
 +
top
 +
list logged on users, and what they are doing
 +
w
 +
shows processes
 +
ps
 +
list open files, by process
 +
lsof
 +
info about processors
 +
cat /proc/cpuinfo
 +
nb of processor on system
 +
grep -ic ^processor /proc/cpuinfo
  
 
= Running scripts =
 
= Running scripts =
Line 64: Line 87:
 
Remove 10 first line of a file
 
Remove 10 first line of a file
 
  sed '1,10d' myfile
 
  sed '1,10d' myfile
 +
 +
Checking file type :
 +
file myfile
 +
 +
Converting dos "end-like" file to unix
 +
  perl -p -e 's/\r$//' < myfile > mynewfile
 +
 +
Checking ascii content of a file :
 +
od -c myfile | more
 +
 +
Intersect 2 files
 +
comm -12 a b
 +
 +
Substraction (lines unique to a)
 +
comm -23 a b
 +
 +
Symmetric difference
 +
comm -2 a b

Latest revision as of 17:18, 9 November 2010

Armand's unix memo

Disc checks

How to find which partition is which ?

sudo fdisk -l /dev/hda

How to check a disk

sudo fsck /dev/sda1

How to find the bad blocks

sudo badblocks /dev/sda1

How to reformat a disc ignoring bad blocks

sudo mke2fs -c /dev/sda1

R tricks

R locale setting for Mac OS X

To enforce US-english setting regardless of the system setting :

system("defaults write org.R-project.R force.LANG en_US.UTF-8")

Note that you must always use .UTF-8 version of the locale, otherwise R.app will not work properly..

System monitoring

list processes

top

list logged on users, and what they are doing

w

shows processes

ps

list open files, by process

lsof

info about processors

cat /proc/cpuinfo

nb of processor on system

grep -ic ^processor /proc/cpuinfo

Running scripts

How to execute a given command for many different parameters stored in a file

cat paramList.txt | xargs mycommand


SSH

How to generate public key

ssh-keygen -f dsa

How to display variables for sshagent

ssh-agent

How to create a passphrase

ssh-add

How to authorize ssh connection to a remote machine using the generated key

cat .ssh/id_dsa.pub | ssh machine_name "cat - >> .ssh/authorized_keys"


File manipulation

How to print a section of a file

awk 'NR >= mystart && NR <= myend' myfile

How to count #of columns per line

awk '{ print NF }' myfile

Print a given column (i.e. 2nd one)

awk '{ print $2 }' myfile
cut -f2 myfile

Pasting 2 files together by their columns

paste file1 file2

Joining 2 files by a common column (ie 1st column of file1 contains some common identifiers than the 3rd column of file2)

join -1 1 -2 3 file1 file2

Sorting numerically a file by its 3rd column

sort -n +2 myfile

Sorting numerically a file by its 2nd column then 1st and then 3rd

sort -n -k 2,1,3 myfile

Splitting a file into smaller files with a fixed number of lines (i.e. 100)

split -l 100 myfile

Remove 10 first line of a file

sed '1,10d' myfile

Checking file type :

file myfile

Converting dos "end-like" file to unix

 perl -p -e 's/\r$//' < myfile > mynewfile

Checking ascii content of a file :

od -c myfile | more

Intersect 2 files

comm -12 a b

Substraction (lines unique to a)

comm -23 a b

Symmetric difference

comm -2 a b