Option to place workspace and service storage in RAM #2299
Replies: 2 comments
|
Would be nice also to have an option to use ram disk for service storage (i.e. /var/lib/mysql in MariaDB containers). |
|
RAM disk for workspace storage is a great optimization for build pipelines — disk I/O is often the bottleneck for build tasks that do a lot of small file operations (node_modules, Go build cache, Rust target/). The performance case:
For Go builds where the compiler reads thousands of small object files, the IOPS difference matters more than bandwidth. Implementation approaches: # Option 1: Mount tmpfs before pipeline runs
steps:
setup:
image: alpine
commands:
- mkdir -p /mnt/workspace
- mount -t tmpfs -o size=4G tmpfs /mnt/workspace
privileged: true
build:
image: golang:1.22
volumes:
- /mnt/workspace:/workspace
commands:
- cd /workspace
- go build ./...# Option 2: Use Docker's tmpfs mount (host handles allocation)
steps:
build:
image: golang:1.22
volumes:
- type: tmpfs
target: /workspace
tmpfs:
size: 4294967296 # 4GB in bytesThe tradeoff: RAM is limited and not shared between concurrent pipelines. For large workspaces (node_modules can be 1-2GB), RAM disk is expensive. Selective use — only the build cache dir, not the full workspace — often gives 80% of the benefit at lower cost. For Woodpecker specifically: would this be a volume type configuration, or a pipeline-level annotation? |
Uh oh!
There was an error while loading. Please reload this page.
For storage intensive pipelines would be nice to have an option to put workspace in ram disk (i.e. tmpfs); will be huge relief for systems with plenty of RAM and slower storage; pipelines will probably be much faster too.
Similar drone issue:
https://discourse.drone.io/t/make-drone-src-workspace-a-tmpfs/8040
Regards,
Paweł
All reactions