Difference between revisions of "UNIX recipes"
(5 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | + | Armand's unix memo | |
− | + | = Disc checks = | |
How to find which partition is which ? | How to find which partition is which ? | ||
Line 15: | Line 15: | ||
sudo mke2fs -c /dev/sda1 | sudo mke2fs -c /dev/sda1 | ||
+ | = R tricks = | ||
− | == Running scripts | + | R locale setting for Mac OS X |
+ | |||
+ | To enforce US-english setting regardless of the system setting : | ||
+ | system("defaults write org.R-project.R force.LANG en_US.UTF-8") | ||
+ | |||
+ | Note that you must always use .UTF-8 version of the locale, otherwise R.app will not work properly.. | ||
+ | |||
+ | = System monitoring = | ||
+ | |||
+ | list processes | ||
+ | top | ||
+ | list logged on users, and what they are doing | ||
+ | w | ||
+ | shows processes | ||
+ | ps | ||
+ | list open files, by process | ||
+ | lsof | ||
+ | info about processors | ||
+ | cat /proc/cpuinfo | ||
+ | nb of processor on system | ||
+ | grep -ic ^processor /proc/cpuinfo | ||
+ | |||
+ | = Running scripts = | ||
How to execute a given command for many different parameters stored in a file | How to execute a given command for many different parameters stored in a file | ||
cat paramList.txt | xargs mycommand | cat paramList.txt | xargs mycommand | ||
− | + | ||
+ | = SSH = | ||
How to generate public key | How to generate public key | ||
Line 36: | Line 60: | ||
− | + | = File manipulation = | |
How to print a section of a file | How to print a section of a file | ||
Line 63: | Line 87: | ||
Remove 10 first line of a file | Remove 10 first line of a file | ||
sed '1,10d' myfile | sed '1,10d' myfile | ||
+ | |||
+ | Checking file type : | ||
+ | file myfile | ||
+ | |||
+ | Converting dos "end-like" file to unix | ||
+ | perl -p -e 's/\r$//' < myfile > mynewfile | ||
+ | |||
+ | Checking ascii content of a file : | ||
+ | od -c myfile | more | ||
+ | |||
+ | Intersect 2 files | ||
+ | comm -12 a b | ||
+ | |||
+ | Substraction (lines unique to a) | ||
+ | comm -23 a b | ||
+ | |||
+ | Symmetric difference | ||
+ | comm -2 a b |
Latest revision as of 17:18, 9 November 2010
Armand's unix memo
Disc checks
How to find which partition is which ?
sudo fdisk -l /dev/hda
How to check a disk
sudo fsck /dev/sda1
How to find the bad blocks
sudo badblocks /dev/sda1
How to reformat a disc ignoring bad blocks
sudo mke2fs -c /dev/sda1
R tricks
R locale setting for Mac OS X
To enforce US-english setting regardless of the system setting :
system("defaults write org.R-project.R force.LANG en_US.UTF-8")
Note that you must always use .UTF-8 version of the locale, otherwise R.app will not work properly..
System monitoring
list processes
top
list logged on users, and what they are doing
w
shows processes
ps
list open files, by process
lsof
info about processors
cat /proc/cpuinfo
nb of processor on system
grep -ic ^processor /proc/cpuinfo
Running scripts
How to execute a given command for many different parameters stored in a file
cat paramList.txt | xargs mycommand
SSH
How to generate public key
ssh-keygen -f dsa
How to display variables for sshagent
ssh-agent
How to create a passphrase
ssh-add
How to authorize ssh connection to a remote machine using the generated key
cat .ssh/id_dsa.pub | ssh machine_name "cat - >> .ssh/authorized_keys"
File manipulation
How to print a section of a file
awk 'NR >= mystart && NR <= myend' myfile
How to count #of columns per line
awk '{ print NF }' myfile
Print a given column (i.e. 2nd one)
awk '{ print $2 }' myfile cut -f2 myfile
Pasting 2 files together by their columns
paste file1 file2
Joining 2 files by a common column (ie 1st column of file1 contains some common identifiers than the 3rd column of file2)
join -1 1 -2 3 file1 file2
Sorting numerically a file by its 3rd column
sort -n +2 myfile
Sorting numerically a file by its 2nd column then 1st and then 3rd
sort -n -k 2,1,3 myfile
Splitting a file into smaller files with a fixed number of lines (i.e. 100)
split -l 100 myfile
Remove 10 first line of a file
sed '1,10d' myfile
Checking file type :
file myfile
Converting dos "end-like" file to unix
perl -p -e 's/\r$//' < myfile > mynewfile
Checking ascii content of a file :
od -c myfile | more
Intersect 2 files
comm -12 a b
Substraction (lines unique to a)
comm -23 a b
Symmetric difference
comm -2 a b