ubuntu 20.04 LTS Server initial setting for machine learning with GPU

Ubuntu 20.04 LTS:focal Initial setup

• written by Young Jin Kim(kimyoungjin06@gmail.com), 2020.09.11
• Add Nvidia driver install simple option, 2021.01.25
  [TOC]

Essential Packages, Apps and Setting

Korean(Hangul) setting

• set with this link
• must "Hangul", not just "Korean"

gnome-tweaks and others

• set with this link
• gnome-tweaks, nautilus-admin, gnome-sushi, chrome

Typora

• set with official page
• It is most powerful markdown editor

Docker

• set with official docs
• All of all!

Nvidia graphic driver

• (Option1) set with this link

  • This is so cool(simple)!

    sudo ubuntu-drivers autoinstall
    sudo reboot

• (Option2) set with this link

  sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
  sudo sh -c 'echo "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 /" >> /etc/apt/sources.list.d/cuda.list'
  sudo sh -c 'echo "deb http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 /" >> /etc/apt/sources.list.d/cuda.list'
  sudo apt-get update
  sudo apt-get install nvidia-driver-450

• CHECK the install

  • Driver Version and CUDA Version is showing, it's success!

    nvidia-smi

Additional setting: for use Jupyter with GPU

nvidia-docker

add linux repository

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
  sudo apt-key add -

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)

curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-docker.list

install nvidia-docker2 and re-load docker demon

sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo pkill -SIGHUP dockerd

(Option1)Get nvidia pytorch image

• From this, check the image with contains list.
• And Check the version of nvidia pytorch

sudo docker pull nvcr.io/nvidia/pytorch:20.12-py3

(Option2)Get tensorflow:latest-devel-gpu

sudo docker pull tensorflow/tensorflow:latest-devel-gpu

Run container with useful options

• Refer this

• (Option1) nvidia pytorch

  • Add cache memory with --ipc=host
  • In this container, base position is /workspace with some example, So, change the base position for volume share

sudo nvidia-docker run -it -p 8888:8888 -p 6006:6006 -h jupyter -v /home/kimyoungjin06/Workspace:/workspace/Workspace --name jupy --ipc=host nvcr.io/nvidia/pytorch:20.12-py3

• (Option2) tensorflow

sudo nvidia-docker run -it -p 8888:8888 -p 6006:6006 -h jupyter -v /home/Workspace:/home/Workspace --name jupyter tensorflow/tensorflow:latest-devel-gpu

Setting internal container

• install jupyter and jupyterlab

pip install jupyter jupyterlab
jupyter lab --generate-config
apt-get install -y vim
vi /root/.jupyter/jupyter_notebook_config.py

• set configure

c.NotebookApp.allow-root = True
c.NotebookApp.token = ''
c.NotebookApp.ip = '*'
c.NotebookApp.notebook_dir = '/home/Workspace'

\*\*\* 왜인지 모르겠으나, 위 내용을 configure file 맨 앞에 쓰면 실행이 되지 않고, 각 항목의 위치에 적어야 제대로 실행됨, 원인 찾는 중

• if install tensorflow/tensorflow, add export cmd in bashrc

  vi /etc/bash.bashrc
  # Add following cmd
  export LD_LIBRARY_PATH=/usr/local/cuda/lib64/

• start jupyter lab

jupyter lab&

• Ctrl + p, q (Exit without close container)

Additional setting: Virtualbox (using window)

• set with this link 1 2 3
• for windows using Office365 or Hwp and other Korean System