feat: dev container support by sora-mono · Pull Request #248 · clice-io/clice

sora-mono · 2025-09-08T16:39:51Z

Currently I only finished linux Dockerfile and launch scripts

As for the plan, there is still a lot work to do:

Support Linux and Windows dev-container
Support Docker image build CI/CD, with cache to acclerate build speed
Move all package version and hash away from cmake and xmake to separated config folder, which is easy for share and manage between different systems
Support toolchain version config
Modify CI to build everything with docker
Test all build function could work in docker image

How to try this preview version:
cd clice # clice project root dir
docker/linux/build.sh
docker/linux/run.sh

miyanyan · 2025-09-09T01:05:56Z

you can add uv to the dev container so the users can run tests

aurora0x27 · 2025-09-09T05:05:51Z

There is a space in front of the .dockerignore file, is this intentional?

sora-mono · 2025-09-09T13:00:44Z

you can add uv to the dev container so the users can run tests

@miyanyan Thank you for your advice, I will convert from apt to uv for python and python library management

miyanyan · 2025-09-09T13:11:14Z

you can add uv to the dev container so the users can run tests

@miyanyan Thank you for your advice, I will convert from apt to uv for python and python library management

Yeah, they use uv for test

clice/.github/workflows/cmake.yml

Line 94 in 2ffab4f

    
                     uv run pytest -s --log-cli-level=INFO tests/integration --executable=./build/bin/clice.exe

sora-mono · 2025-09-09T13:13:56Z

There is a space in front of the .dockerignore file, is this intentional?

@aurora0x27 Thank you for your interest!
I'm not sure which step generated the .dockerignore file.
As for my design, I'm not mean to create it and not planning to exclude any file from local workspace. Mounting the whole clice project directory to docker container won't cost more than exclude some files, but excluding folders could confuse user when they modified something from excluded path in the container.
If you found docker generated the .dockerignore file, just remove it.
If it keeps on generated, I'll commit an empty .dockerignore to supress the generation

16bit-ykiko · 2025-09-09T13:17:49Z

There is an extra space in dockignore filename https://github.com/clice-io/clice/blob/main/%20.dockerignore, could you also fix it?

sora-mono · 2025-09-09T13:20:01Z

There is an extra space in dockignore filename https://github.com/clice-io/clice/blob/main/%20.dockerignore, could you also fix it?

Sure, I'll fix it together

sora-mono · 2025-09-09T18:06:16Z

Update:

python management changed from apt to uv
Parallel dependency install and download support
docker buildx cache for llvm-binary download
bug fix

sora-mono · 2025-09-13T16:39:13Z

Update:

XMake package cache support in Dockerfile
Install exactly only one toolchain
Fixed async crash in dockerfile
Refactored package install sequence and source for build speed and avoid other compile tool chain installation

clang toolchain and xmake cache are ready
However lld would report unresolved symbols, which is still on inspecting

sora-mono · 2025-09-14T16:22:17Z

[MileStone]

Linux dev-container support is ready for use.

Compiler support for Clang and GCC, clang could pass compile and test, gcc is able to compile but still have compile error to fix(Not caused by dev-container).
By default, clang is selected for image. If you want to try gcc, run with:
./build.sh --compiler gcc && ./run.sh --compiler gcc
[Currently Fallback] Caching llvm-binary is currently disabled, as current implement would create 10G size layer, which is too big for build and pull.
I have to find a better way to cache only llvm-binary .tar.xz.

sora-mono · 2025-09-16T15:43:39Z

After discussions with @16bit-ykiko @star-hengxing, Stage1 dev-container will be a preview feature.
Stage2 will support for static linking libstdc++, which requires self compiled toolchain, so I have to do a refactor for Docker image building.
Finally, we will use dev container to provide a stable and consistent build environment, keeping the dev environment the same with CI and release environment, which would completely solve problems caused by inconsistent build environment.
Windows dev container will be supported after Linux container refactor.
CI/CD is scheduled after the Windows dev container.

1. python management changed from apt to uv 2. Parallel dependency install and download support 3. docker buildx cache for llvm-binary download 4. bug fix

…ke.lua refactor done) 2. Install exactly only one toolchain 3. Fixed async crash in dockerfile 4. Refactored package install sequence and source for build speed and avoid other compile tool chain installation

preinstall libstdc++, instead of libc++

Preparing to merge Stage1 done

Optimized xmake installation for faster and less download size Optimized xmake change readability

sora-mono · 2025-09-18T11:51:37Z

@16bit-ykiko Ready for merge, requesting review

Removed useless cache options from scripts

16bit-ykiko · 2025-09-18T14:58:00Z

/gemini summary

16bit-ykiko · 2025-09-18T14:58:10Z

/gemini review

gemini-code-assist · 2025-09-18T14:58:22Z

Summary of Changes

This pull request introduces the foundational elements for development container support, specifically for Linux environments. The primary goal is to establish a consistent and reproducible build environment, simplifying setup for developers. It includes a Dockerfile for setting up a Linux development image with configurable compilers (GCC or Clang) and scripts to build and run these containers. Additionally, the changes centralize toolchain version definitions and enhance the project's build system to better manage external dependencies like LLVM.

Highlights

Linux Dev Container Support: Initial support for Linux development containers has been added, providing a standardized environment for building and developing the project. This includes a Dockerfile and associated build/run scripts.
Toolchain Version Management: Toolchain versions for xmake, cmake, python, gcc, and clang are now centralized in a new configuration file (config/default-toolchain-version), making it easier to manage and share versions across different systems.
Refactored LLVM Package Management: The xmake build system's handling of LLVM dependencies has been refactored to dynamically load prebuilt binary information from a JSON configuration file (config/prebuilt-llvm.json), improving flexibility and maintainability.
Python Package Management with uv: The Dockerfile now uses uv for Python installation and package management, replacing apt for a more efficient and consistent approach.
Docker Build Cache Improvements: The Dockerfile incorporates multi-stage builds and BuildKit cache mounts for apt, xmake, cmake, and uv to accelerate build speeds and improve cache efficiency.

Changelog

.dockerignore
- Updated to include more build artifacts and IDE-related files (out/, .clice/, .llvm*/, .xmake/, .vscode/, .vs/) to prevent them from being copied into the Docker image.
.gitignore
- Added .vs/ to the ignore list for Visual Studio related files.
config/default-toolchain-version
- New file added to define default versions for various tools and compilers (xmake, cmake, python, gcc, clang, msvc).
config/prebuilt-llvm.json
- New file added containing JSON metadata for prebuilt LLVM binaries, including platform, build type, LTO status, filename, version, and SHA256 checksums.
docker/linux/Dockerfile
- New Dockerfile implementing a multi-stage build process for a Linux development container.
- Configures DEBIAN_FRONTEND to noninteractive and adds /root/.local/bin to PATH.
- Installs basic tools like software-properties-common, curl, gnupg, git, xz-utils, unzip, and make.
- Adds GCC and LLVM PPAs.
- Includes a compiler stage to install libstdc++ development files and either GCC or Clang based on a build argument, managing alternatives for cc, gcc, c++, g++, clang, clang++, and ld.
- Features a build-tool stage that installs xmake, cmake, and python (via uv) with caching for downloads and installations.
- Includes a dependency-cache stage (currently commented out for xmake packages) and a final stage to clean up and set the working directory.
docker/linux/build.sh
- New script to build the Linux development container Docker image.
- Allows specifying the compiler (gcc or clang) via --compiler argument.
- Sets the image tag and name based on the selected compiler.
- Uses docker buildx build with build arguments for compiler and project root, and progress output.
docker/linux/run.sh
- New script to run or attach to the Linux development container.
- Allows specifying the compiler (gcc or clang) via --compiler argument.
- Includes a --reset option to stop and remove an existing container.
- Automatically builds the image if it doesn't exist by invoking build.sh.
- Checks for an existing container and attaches to it if found, otherwise creates and runs a new one with bind mounts for the project directory.
docs/en/dev/build.md
- Added a new section 'Development Container' with instructions on how to build, run, and reset the Linux dev container using the new scripts.
- Included a note indicating the feature is in preview and currently Linux-only.
docs/zh/dev/build.md
- Added a new section 'Dev Container' with instructions in Chinese on how to use the new Linux dev container scripts.
- Included a note in Chinese about the preview status and future plans.
xmake.lua
- Changed add_requires("llvm", {system = false}) to add_requires("clice-llvm", {alias = "llvm"}).
- Updated the clice_build_config rule to apply -fno-rtti cxflags for clangxx and gxx tools in addition to clang and gcc.
- Renamed the package("llvm") definition to package("clice-llvm").
- Modified the clice-llvm package's on_source function to read LLVM binary download information (URLs and SHA256 checksums) from config/prebuilt-llvm.json based on platform, build type, and LTO status, replacing hardcoded values.

Activity

miyanyan suggested adding uv to the dev container for running tests.
aurora0x27 identified an unintended space in the .dockerignore filename.
sora-mono acknowledged miyanyan's suggestion and confirmed plans to switch Python management from apt to uv.
miyanyan provided a link to demonstrate uv usage in existing workflows.
sora-mono responded to aurora0x27, explaining uncertainty about the .dockerignore generation and confirming a fix.
16bit-ykiko also pointed out the extra space in the .dockerignore filename and requested a fix.
sora-mono confirmed that the .dockerignore issue would be fixed.
sora-mono provided an update on the PR, detailing changes including the switch to uv for Python, parallel dependency installation, Docker buildx cache for LLVM, and bug fixes.
sora-mono gave another update, highlighting XMake package cache support, exact toolchain installation, a fix for an async crash, and refactoring of package installation sequence.
sora-mono announced that Linux dev-container support is ready for use, noting compiler support (Clang working, GCC compiling but with errors), and a temporary disablement of LLVM binary caching due to large layer size.
sora-mono discussed with 16bit-ykiko and star-hengxing that Stage1 dev-container would be a preview feature, with future plans for static linking, Windows support, and CI/CD integration.
sora-mono requested a review from 16bit-ykiko, indicating the PR was ready for merge.
star-hengxing provided a review comment suggesting the use of add_requires("clice-llvm", {alias = "llvm"}) and add_packages("llvm").
16bit-ykiko commented on the documentation, requesting a space between Chinese and English text.
sora-mono confirmed that the documentation spacing issue was fixed.

gemini-code-assist

Code Review

This pull request introduces excellent support for development containers, which will significantly streamline the setup process for new developers. The use of a multi-stage Dockerfile, build scripts, and the refactoring of xmake.lua to be data-driven are all great improvements. I've provided a few suggestions to enhance the robustness and maintainability of the new scripts and Dockerfile, focusing on removing redundant code, cleaning up log output, addressing a potentially risky bind mount, and clarifying a change in the xmake build policy.

miyanyan · 2025-09-19T03:25:21Z

There are two docker files, the Dockerfile(dev, linux + windows finally) in this pr, and https://github.com/clice-io/clice/blob/main/Dockerfile for release, is maintaining the two docker files the right way? It seems they have the same points, only the docker stage they are in is different.

16bit-ykiko · 2025-09-19T12:35:15Z

Remove this file https://github.com/clice-io/clice/blob/main/Dockerfile?

sora-mono · 2025-09-19T13:13:25Z

@16bit-ykiko @miyanyan I plan to replace all builds including CI with dev-container, but not now.
Also, this PR only contains Linux platform, and it will be refactored to support static linking libstdc++.
Althouth not completely supported, I decided to merge only Linux because someone requested for this feature merged as quickly as possible, it could be very annoying for them to prepare toolchain in Linux, so this PR would be a PREVIEW feature and not complete resolution.
As for Windows platform toolchain, it is easy to install and use(I fixed msvc build in another PR and those who build on windows could build with the latest msvc toolchain, without the need to install clang)
So the Dockerfile in root should not be removed immediately, there's already a github workflow for it, and could be removed until this PR is completely supported.

sora-mono mentioned this pull request Sep 8, 2025

Build with docker #191

Closed

miyanyan mentioned this pull request Sep 9, 2025

feat: add vscode devcontainer #247

Closed

sora-mono changed the title ~~Preview for Linux dev-container, not completely tested~~ Dev Container Support Sep 9, 2025

sora-mono force-pushed the feature/dev-container branch from 7a69af0 to 12e4fb7 Compare September 13, 2025 16:36

sora-mono force-pushed the feature/dev-container branch from 6de7172 to f12efbc Compare September 14, 2025 14:16

star-hengxing reviewed Sep 17, 2025

View reviewed changes

Comment thread xmake.lua Outdated

sora-mono marked this pull request as ready for review September 17, 2025 15:24

sora-mono added 12 commits September 17, 2025 23:24

Preview for Linux dev-container, not completely tested

4e543da

Refactored Dockerfile for LInux

f36a4d4

1. python management changed from apt to uv 2. Parallel dependency install and download support 3. docker buildx cache for llvm-binary download 4. bug fix

1. XMake package cache support in Dockerfile(Not fully work until xma…

7f12f36

…ke.lua refactor done) 2. Install exactly only one toolchain 3. Fixed async crash in dockerfile 4. Refactored package install sequence and source for build speed and avoid other compile tool chain installation

Moved xmake llvm-binary config to config/prebuilt-llvm.json

e40b484

Removed local cache policy

a9b2e57

Format fix

889e0d3

Fix link unsolved symbols

42e9342

preinstall libstdc++, instead of libc++

Fix format check

76d212a

Support persist container

7a22c38

Preparing to merge Stage1 done

Updated Document

973cbf2

Revert base image system version

8253707

Optimized xmake installation for faster and less download size Optimized xmake change readability

Format Fix

87a9e21

sora-mono force-pushed the feature/dev-container branch from a7d8252 to 87a9e21 Compare September 17, 2025 15:24

fix xmake rtti flags

f3bb876

16bit-ykiko reviewed Sep 18, 2025

View reviewed changes

Comment thread docs/zh/dev/build.md Outdated

Document format fix

0f130b3

Removed useless cache options from scripts

gemini-code-assist Bot reviewed Sep 18, 2025

View reviewed changes

Comment thread docker/linux/Dockerfile

Comment thread docker/linux/Dockerfile

Comment thread docker/linux/build.sh

Comment thread docker/linux/run.sh

Comment thread xmake.lua

star-hengxing approved these changes Sep 19, 2025

View reviewed changes

16bit-ykiko approved these changes Sep 19, 2025

View reviewed changes

16bit-ykiko merged commit f94927d into clice-io:main Sep 19, 2025
6 checks passed

16bit-ykiko changed the title ~~Dev Container Support~~ feat: dev container support Apr 6, 2026

Conversation

sora-mono commented Sep 8, 2025

Uh oh!

miyanyan commented Sep 9, 2025

Uh oh!

aurora0x27 commented Sep 9, 2025

Uh oh!

sora-mono commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

miyanyan commented Sep 9, 2025

Uh oh!

sora-mono commented Sep 9, 2025

Uh oh!

16bit-ykiko commented Sep 9, 2025

Uh oh!

sora-mono commented Sep 9, 2025

Uh oh!

sora-mono commented Sep 9, 2025

Uh oh!

sora-mono commented Sep 13, 2025

Uh oh!

sora-mono commented Sep 14, 2025

Uh oh!

sora-mono commented Sep 16, 2025

Uh oh!

Uh oh!

sora-mono commented Sep 18, 2025

Uh oh!

Uh oh!

16bit-ykiko commented Sep 18, 2025

Uh oh!

16bit-ykiko commented Sep 18, 2025

Uh oh!

gemini-code-assist Bot commented Sep 18, 2025

Summary of Changes

Highlights

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

miyanyan commented Sep 19, 2025

Uh oh!

16bit-ykiko commented Sep 19, 2025

Uh oh!

sora-mono commented Sep 19, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sora-mono commented Sep 9, 2025 •

edited

Loading