• chmod
• find
• rsync
• nohup
• pbcopy
• xclilp
• Print Information
• Image
• Download Things
  • pypi
  • YouTube
• Hardware Info
• Compile Things
• CUDA Related
  • Install
  • Unknown Error
• HPC Related
  • Transfer files
  • Use More Memory

chmod

Example,

• chmod +x FILENAME: give executable permission for file named FILENAME for all users
• chmod 664 FILENAME: change the permission for FILENAME to -rw-rw-r--, meaning
  • write and read, rw, for owner
  • write and read, rw, for group member
  • read only, r, for other users.

Some useful octal notations are in the table.

find

# Find files by name
find /path/to/search -name "filename.txt"

# Case insensitive search
find /path/to/search -iname "filename.txt"

# Find directories only
find /path/to/search -type d -name "directory_name"

# Find files based on file size (e.g., larger than 50MB)
find /path/to/search -type f -size +50M

# Find files and execute a command on each file (e.g., list details)
find /path/to/search -name "*.txt" -exec ls -lh {} \;

# Find files modified in the last 7 days
find /path/to/search -mtime -7

# Find and delete files (e.g., files with .bak extension)
find /path/to/search -name "*.bak" -delete

# Find files with specific permissions (e.g., 644)
find /path/to/search -type f -perm 644

# Find files owned by a specific user
find /path/to/search -user username

# Find files that meet multiple conditions (e.g., type file, ends with .txt, larger than 1MB)
find /path/to/search -type f -name "*.txt" -size +1M

# Exclude a specific directory (e.g., node_modules) from the search
find /path/to/search -path /path/to/search/node_modules -prune -o -name "filename.txt" -print

rsync

The rsync command is used for synchronizing files and directories between two locations. The term “rsync” stands for “remote sync”.

rsync [OPTIONS] SOURCE [SOURCE]... DEST

• rsync: The command itself.
• OPTIONS: Optional flags to specify behavior (e.g., -a for archive mode, -v for verbose).
• SOURCE: The source file(s) or directory(s) you want to copy.
• DEST: The destination where you want to copy the source to.

Example:

rsync -av /path/to/source/ /path/to/destination/

the rsync uses a delta-transfer algorithm to minimize data transfer by only sending the differences between the source and destination files.

Common OPTIONS:

• -r: recursion into directories,
• -p: preserving file permissions
• -a: flag for ‘archive mode’, which is a shorthand for a number of other options (-rlptgoD) that preserve most attributes and enable recursion.

nohup

The nohup command in Linux allows us to run a process in the background and ensures that it keeps running even after the session was logged out. The term “nohup” stands for ‘No Hang Up’.

nohup COMMAND [ARG]... [&]

• COMMAND: The command you want to run.
• ARG: Optional arguments for the command.
• &: Used to run the command in the background.

Example:

nohup python my_script.py &

nohup detaches the process from the terminal and redirects the standard output (stdout) and standard error (stderr) to a file, usually nohup.out.

pbcopy

The pbcopy command is used on macOS systems to copy the standard input (stdin) to the clipboard.

pbcopy < INPUT

• pbcopy: The command itself.
• < INPUT: The input source that you want to copy to the clipboard.

Example:

echo "Hello, world!" | pbcopy

pbcopy reads from stdin and places the contents into the clipboard, allowing you to paste it elsewhere. Typically, you would pipe (|) the output of another command into pbcopy to capture it.

If using ubuntu, put the following in .bashrc, to make a proxy for xclilp.

alias pbcopy='xclip -selection clipboard'

xclilp

The xclip command is used on Linux systems to interface with the X11 clipboard.

xclip [OPTION]... [FILE]...

• xclip: The command itself.
• OPTION: Optional flags to specify behavior (e.g., -sel clip for clipboard).
• FILE: Optional file whose content you want to copy.

Example:

echo "Hello, world!" | xclip -sel clip

Print Information

File size in MB

ls -l --block-size=M
du -sh .

Hash Value

shasum filename
md5 filename
openssl md5

Check OS distribution

cat /etc/os-release

Image

Converting all png pictures into tif format:

mogrify -format tif *.png

Display Image

display picture.png

Convert fonts in PDF to curves

gs -o out.pdf -dNoOutputFonts -sDEVICE=pdfwrite in.pdf

Convert PDF to EPS

gs -q -dNOCACHE -dNOPAUSE -dBATCH -dSAFER -sDEVICE=eps2write\
    -sOutputFile=out.eps in.pdf

Download Things

pypi

Visit https://mirrors.tuna.tsinghua.edu.cn/help/pypi/ to deal with internet connections in China.

Adding

-i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple

after pip install to download from the tsinghua source.

YouTube

Get moderate sized MP4 videos from the Youtube.

yt-dlp \
    -f "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4] / bv*+ba/b"\
    -N 4 \
    

If we see this error

Sign in to confirm you’re not a bot. This helps protect our community. Learn more

This is likey to be because YouTube blocked the current ID and we need a proxy.

Here is the description frou the official GitHub Issue Page.

Your IP has been blocked by YouTube while you are logged out. You can pass logged-in cookies; however, doing so may get your account blocked. The better workaround is to try changing your IP

Hardware Info

• get CPU information: lscpu
• get GPU information: nvidia-smi
• measure memory bandwidth: mbw

Compile Things

Configuring Vim:

./configure --with-features=huge \
            --enable-multibyte \
            --enable-rubyinterp=yes \
            --enable-python3interp=yes \
            --with-python3-config-dir=$(python3-config --configdir) \
            --enable-cscope \
            --prefix=/usr/local

Configuring Python 3.8

sudo rm -rf Python-3.8.6
tar -xzf Python-3.8.6.tgz
cd Python-3.8.6

export LD_LIBRARY_PATH=/usr/local/lib
export LD_RUN_PATH=/usr/local/lib

./configure  --enable-optimizations\
    --prefix=/usr/local\
    --enable-shared\
    --enable-loadable-sqlite-extensions\
	CPPFLAGS="-I/usr/local/include -I/usr/local/include/openssl"\
	LDFLAGS="-L/usr/local/lib -L/usr/local/lib/openssl"

make -j8
sudo make install

cd ..

CUDA Related

Install

the coda toolchain, instead of the driver, is specified by the --toolkit option.

./cuda_11.8.0_520.61.05_linux.run --toolkit

Unknown Error

Several errors can occur because Nvidia kernel is breaks, includeing

• CUDA initialization: CUDA unknown error
• RuntimeError: Torch is not able to use GPU

Solution:

sudo rmmod nvidia_uvm
sudo modprobe nvidia_uvm

Reference

HPC Related

I used BlueCrystal (HPC in the university of Bristol) a lot while doing my PhD. These are the commands I used a lot.

Transfer files

# from local to hpc
scp -r LOCAL_PATH ab12345@bluecrystalp3.bris.ac.uk:REMOTE_PATH

# from local to hpc
scp -r ab12345@bluecrystalp3.bris.ac.uk:REMOTE_PATH LOCAL_PATH 

We can store the username@host into a envirinmental variable to make things faster, by adding the following line into the .basrhc file,

export myhpc="USER_NAME@bluecrystalp3.bris.ac.uk"

After this, we can simplify our code by including $myhpc,

# from local to hpc
scp -r LOCAL_PATH $myhpc:REMOTE_PATH

# from local to hpc
scp -r $myhpc:REMOTE_PATH LOCAL_PATH 

Use More Memory

#PBS -l mem=128gb
#PBS -q  himem