Action Chunking Transformer

What’s Action Chunking Transformer?

In the paper:

We implement action chunking policy with Transformers, an architecture designed for sequence modeling, and train it as a conditional VAE (CVAE)

Architecture

• Input: 여러 카메라 이미지, 로봇 상태(qpos), 환경 상태, (학습 시) 행동 시퀀스
• Backbone: 각 카메라 이미지를 feature로 변환하는 CNN Network
• Transformer: Sequence modeling, 이미지 feature와 로봇 상태, 잠재 변수 등을 입력받음
• VAE Encoder: Action sequence를 latent variable z로 인코딩
• Decoder: latent variable z와 Input을 바탕으로 다음 행동을 예측

How to do?

Github Link: https://github.com/tonyzhaozh/act

Setup

• Clone the repository

  git clone https://github.com/tonyzhaozh/act/tree/main
  

• Create a new conda env

  conda create -n aloha python=3.8.10
  conda activate aloha
  pip install torchvision
  pip install torch
  pip install pyquaternion
  pip install pyyaml
  pip install rospkg
  pip install pexpect
  pip install mujoco==2.3.7
  pip install dm_control==1.0.14
  pip install opencv-python
  pip install matplotlib
  pip install einops
  pip install packaging
  pip install h5py
  pip install ipython
  cd act/detr && pip install -e .
  

Run the script

python3 record_sim_episodes.py \
  --task_name sim_transfer_cube_scripted \
  --dataset_dir ./dataset/ \
  --num_episodes 5  \
  --onscreen_render

Trouble Shooting

OSError: dlopen(/System/Library/OpenGL.framework/OpenGL, 0x0006): tried: '/System/Library/OpenGL.framework/OpenGL' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/System/Library/OpenGL.framework/OpenGL' (no such file), '/System/Library/OpenGL.framework/OpenGL' (no such file, not in dyld cache)

• Edit this:
  /Users/joonhyung-lee/.pyenv/versions/<virtual-env>/lib/python3.10/site-packages/mujoco/cgl/cgl.py

    # 기존
    _CGL = ctypes.CDLL('/System/Library/OpenGL.framework/OpenGL')
  
    # 수정
    _CGL = ctypes.CDLL('/System/Library/Frameworks/OpenGL.framework/OpenGL')
  

실행 예시:

episode_idx=0
Rollout out EE space scripted policy
episode_idx=0 Successful, episode_return=630
Replaying joint commands
episode_idx=0 Successful, episode_return=639
Saving: 0.5 secs

To train ACT

# Transfer Cube task
python3 imitate_episodes.py \
  --task_name sim_transfer_cube_scripted \
  --ckpt_dir ./ckpt/ \
  --policy_class ACT --kl_weight 10 --chunk_size 100 --hidden_dim 512 --batch_size 8 --dim_feedforward 3200 \
  --num_epochs 2000  --lr 1e-5 \
  --seed 0