[TimeSformer, Something-Something v2 설치]

Video Classification Dataset인 Something-Something V2 설치

https://developer.qualcomm.com/software/ai-datasets/something-something

Moving Objects Dataset: Something-Something v. 2

위 사이트에서 회원가입 후에 모든 20BN-Something-Something Download Package를 00~19까지 다운로드
그 뒤에 우분투 서버로 파일은 넣은 뒤에 20BN-Something-Something Download Instructions을 보고 설치

 

TimeSformer에서 보면 초당 30Frame으로 Sampling 해야함

https://github.com/facebookresearch/TimeSformer/blob/main/timesformer/datasets/DATASET.md

GitHub - facebookresearch/TimeSformer: The official pytorch implementation of our paper "Is Space-Time Attention All You Need fo

 

ffmpeg를 이용 경로설정만 알아서 하면 됨

각 Frame당 30으로 셋팅

 

VIDEO_DIR=./20bn-something-something-v2
FRAME_DIR=./frames

for video_file in $VIDEO_DIR/*; do
    video_name=$(basename -- "$video_file")
    video_name="${video_name%.*}"

    mkdir "frame/${video_name}"

    ffmpeg -i "${video_file}" -r 30 -q:v 1 "$FRAME_DIR/${video_name}/${video_name}_%06d.jpg"

done

 

우분투 명령어

폴더개수

ls -l | grep ^d | wc -l

파일 개수

ls -l | grep ^- | wc -l

---

https://hub.docker.com/

Docker Hub Container Image Library | App Containerization

docker image찾기

 

docker images Pull/ TimeSformer Cuda11
docker pull qilf/timesformer_cuda11

 

docker 실행 

docker run --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=0 --volume ~/workspace:/workspace -it --rm --name kimhyunwoo qilf/timesformer_cuda11

 

 

conda create -n timesformer python=3.9 -y
source activate timesformer
pip install torchvision
pip install 'git+https://github.com/facebookresearch/fvcore'
pip install simplejson
pip install einops
pip install timm
conda install av -c conda-forge -y
pip install psutil
pip install scikit-learn
pip install opencv-python
pip install tensorboard
pip install matplotlib
pip install ptflops
pip install torchsummary
cd TimeSformer
python setup.py build develop

python tools/run_net.py  --cfg ./configs/SSv2/TimeSformer_divST_8_224.yaml

videomae
sudo pip install timm==0.4.12
sudo pip install deepspeed==0.5.8
sudo pip install opencv-python
sudo pip install tensorboardX
sudo pip install decord
sudo pip install einops

 

OUTPUT_DIR='./ex1'
# path to SSV2 annotation file (train.csv/val.csv/test.csv)
DATA_PATH='/workspace/ssv2/annotations'
# path to pretrain model
MODEL_PATH='./pretrain/checkpoint.pth'

OMP_NUM_THREADS=1 torchrun --nproc_per_node=2 \
    run_class_finetuning.py \
    --model vit_small_patch16_224 \
    --data_set SSV2 \
    --nb_classes 174 \
    --data_path ${DATA_PATH} \
    --finetune ${MODEL_PATH} \
    --log_dir ${OUTPUT_DIR} \
    --output_dir ${OUTPUT_DIR} \
    --batch_size 6 \
    --num_sample 2 \
    --input_size 224 \
    --short_side_size 224 \
    --save_ckpt_freq 10 \
    --num_frames 16 \
    --opt adamw \
    --lr 2e-3 \
    --layer_decay 0.7 \
    --opt_betas 0.9 0.999 \
    --weight_decay 0.05 \
    --epochs 40 \
    --test_num_segment 2 \
    --test_num_crop 3 \
    --dist_eval \
    
    
OUTPUT_DIR='./ex1'
# path to SSV2 annotation file (train.csv/val.csv/test.csv)
DATA_PATH='/home/work/workspace_kim/ssv2/annotations'
# path to pretrain model
MODEL_PATH='./pretrain/checkpoint.pth'
python run_class_finetuning.py \
    --model vit_small_patch16_224 \
    --data_set SSV2 \
    --nb_classes 174 \
    --data_path ${DATA_PATH} \
    --finetune ${MODEL_PATH} \
    --log_dir ${OUTPUT_DIR} \
    --output_dir ${OUTPUT_DIR} \
    --batch_size 6 \
    --num_sample 2 \
    --input_size 224 \
    --short_side_size 224 \
    --save_ckpt_freq 10 \
    --num_frames 16 \
    --opt adamw \
    --lr 2e-3 \
    --layer_decay 0.7 \
    --opt_betas 0.9 0.999 \
    --weight_decay 0.05 \
    --epochs 40 \
    --test_num_segment 2 \
    --test_num_crop 3 \
    --dist_eval \