codingame_tools.language.toolchain

toolchain

Composable multi-language toolchain images.

One image serves several languages, built from dependency-ordered fragments -- see fragment for the model and why it exists, and subsystems for the shared toolchains languages install onto.

ENV_DIR module-attribute

ENV_DIR = '/opt/cg/env.d'

Where activation scripts live inside the image--see the module docstring.

Under /opt rather than /etc/profile.d deliberately: these are not meant to apply to every shell. Sourcing them all would defeat the point, since two of them may set JAVA_HOME incompatibly. They are opt-in, one at a time, by the command that needs one.

BASE_IMAGE module-attribute

BASE_IMAGE = 'debian:bookworm-slim'

The one neutral base every fragment installs onto.

Deliberately not a language image such as gcc:14. A per-language base cannot compose--two of them cannot both be FROM--and it hides the toolchain version in an image tag, which is exactly how the C++ build came to be silently two major gcc releases ahead of CodinGame's.

PREAMBLE module-attribute

PREAMBLE = '# Common to every cg toolchain image, whatever languages it carries.\nENV DEBIAN_FRONTEND=noninteractive\nRUN apt-get update \\\n    && apt-get install -y --no-install-recommends ca-certificates coreutils \\\n    && rm -rf /var/lib/apt/lists/*\nRUN mkdir -p /opt/cg/env.d /build\nWORKDIR /build\n'

Statements shared by every image, before any fragment.

coreutils supplies the timeout and stdbuf the run and debug paths depend on (already present on Debian; named so a swapped base still gets them).

SUBSYSTEMS module-attribute

SUBSYSTEMS = (_GCC11, _PYTHON311, _JDK21, _DOTNET8, _NODE20)

Every subsystem cg ships. Collected by codingame_tools.language.toolchain.registry.

CgToolchainError

Bases: Exception

Raised for an unresolvable fragment set--an unknown slug, or a dependency cycle.

CgToolchainFragment dataclass

CgToolchainFragment(slug, version, depends_on=(), dockerfile='', env_script='')

One composable piece of a toolchain image.

slug instance-attribute

slug

Stable identifier, used in dependency edges, in the generated header, and as the activation script's filename. Lowercase, no spaces--it appears in shell and Dockerfile contexts.

version instance-attribute

version

Bumped whenever dockerfile or env_script changes, so an unmodified generated Dockerfile can be detected as stale and regenerated. Deliberately per fragment rather than one global template version: changing the Rust fragment shouldn't invalidate a C++-only image.

depends_on class-attribute instance-attribute

depends_on = ()

Slugs that must be installed before this one. The mechanism that lets several languages share a toolchain (C and C++ -> gcc11) and lets conflicting ones coexist (java -> jdk21 while scala -> jvm8).

dockerfile class-attribute instance-attribute

dockerfile = ''

Statements inserted verbatim. Legitimately empty: a language whose toolchain is entirely supplied by a subsystem contributes only its dependency edge and its activation script, and emits nothing here. An empty fragment produces no Dockerfile section and so no extra layer.

env_script class-attribute instance-attribute

env_script = ''

Body of /opt/cg/env.d/<slug>.sh, if this fragment needs one. The composer prepends the . <dep>.sh lines itself, so a fragment only writes its own exports.

render_dockerfile

render_dockerfile(fragments, *, base_image, preamble='')

The full cg-owned Dockerfile for fragments, already in install order.

Renders cg's own base.dockerfile content. Distinct from codingame_tools.language._docker.compose_dockerfile, which composes that file on disk with the user's custom.dockerfile -- generation versus merging.

The header is machine-readable in the same spirit as the single-language one it replaces: it records every fragment and its version, so a generated file can be recognized as cg's, checked for staleness, and told apart from one the user has edited.

Parameters:

• fragments (list[CgToolchainFragment]) –

  In install order, as returned by resolve_fragments.

• base_image (str) –

  Value for the CG_BASE_IMAGE build arg--one pinned neutral base that every fragment installs onto, rather than a per-language base image.

• preamble (str, default: '' ) –

  Statements common to every image, inserted before any fragment.

Source code in codingame_tools/language/toolchain/fragment.py

def render_dockerfile(
            fragments: list[CgToolchainFragment],
            *,
            base_image: str,
            preamble: str = "",
        ) -> str:
    """The full cg-owned Dockerfile for `fragments`, already in install order.

       *Renders* cg's own `base.dockerfile` content. Distinct from
       `codingame_tools.language._docker.compose_dockerfile`, which *composes* that file on disk with
       the user's `custom.dockerfile` -- generation versus merging.

       The header is machine-readable in the same spirit as the single-language one it replaces: it
       records every fragment and its version, so a generated file can be recognized as cg's, checked
       for staleness, and told apart from one the user has edited.

    Args:
        fragments:  In install order, as returned by `resolve_fragments`.
        base_image: Value for the `CG_BASE_IMAGE` build arg--one pinned neutral base that every
                     fragment installs onto, rather than a per-language base image.
        preamble:   Statements common to every image, inserted before any fragment.
    """
    manifest = ",".join(f"{f.slug}@{f.version}" for f in fragments)
    body_parts: list[str] = [
        f"ARG CG_BASE_IMAGE={base_image}\n",
        "FROM ${CG_BASE_IMAGE}\n",
    ]
    if preamble.strip():
        body_parts.append("\n" + preamble.strip("\n") + "\n")
    for fragment in fragments:
        section = fragment.dockerfile.strip("\n")
        env = _env_script_statements(fragment)
        if not section and not env:
            # Normal, not degenerate: a language wholly supplied by a subsystem. Emitting an empty
            # section would add a comment-only layer and, worse, make two identical images differ.
            continue
        body_parts.append(f"\n# --- {fragment.slug} ---\n")
        if section:
            body_parts.append(section + "\n")
        if env:
            body_parts.append(env)
    body = "".join(body_parts)

    header = (
            "# cg-managed toolchain--do not edit.\n"
            "# Put your own additions in custom.dockerfile instead; they're appended to this file\n"
            "# and survive every cg template upgrade.\n"
            f"# cg-toolchain: fragments={manifest} "
            f"body-sha256={hashlib.sha256(body.encode('utf-8')).hexdigest()}\n"
        )
    return header + body

resolve_fragments

resolve_fragments(requested, registry)

Every fragment needed for requested, dependencies first, in deterministic order.

Ties are broken by slug so the result is stable across runs and, more importantly, so a subset's order is a prefix of a superset's--see the module docstring on layer sharing.

Parameters:

• requested (Iterable[str]) –

  Slugs asked for, in any order. Duplicates are harmless.

• registry (Mapping[str, CgToolchainFragment]) –

  Every known fragment, keyed by slug.

Returns:

• list[CgToolchainFragment] –

  Fragments in install order.

Raises:

• CgToolchainError –

  on an unknown slug or a dependency cycle.

Source code in codingame_tools/language/toolchain/fragment.py

def resolve_fragments(
            requested: Iterable[str],
            registry: Mapping[str, CgToolchainFragment],
        ) -> list[CgToolchainFragment]:
    """Every fragment needed for `requested`, dependencies first, in deterministic order.

       Ties are broken by slug so the result is stable across runs and, more importantly, so a
       subset's order is a prefix of a superset's--see the module docstring on layer sharing.

    Args:
        requested: Slugs asked for, in any order. Duplicates are harmless.
        registry:  Every known fragment, keyed by slug.

    Returns:
        Fragments in install order.

    Raises:
        CgToolchainError: on an unknown slug or a dependency cycle.
    """
    table = dict(registry)

    def lookup(slug: str) -> CgToolchainFragment:
        try:
            return table[slug]
        except KeyError:
            known = ", ".join(sorted(table)) or "<none>"
            raise CgToolchainError(
                    f"unknown toolchain fragment {slug!r}. Known: {known}") from None

    ordered: list[CgToolchainFragment] = []
    done: set[str] = set()
    # Depth-first with an explicit "in progress" set, so a cycle is reported with the path that
    # closes it rather than as a bare RecursionError.
    visiting: list[str] = []

    def visit(slug: str) -> None:
        fragment = lookup(slug)
        if fragment.slug in done:
            return
        if fragment.slug in visiting:
            cycle = " -> ".join([*visiting[visiting.index(fragment.slug):], fragment.slug])
            raise CgToolchainError(f"toolchain fragment dependency cycle: {cycle}")
        visiting.append(fragment.slug)
        for dependency in sorted(fragment.depends_on):
            visit(dependency)
        visiting.pop()
        done.add(fragment.slug)
        ordered.append(fragment)

    for slug in sorted(set(requested)):
        visit(slug)
    return ordered

all_fragments

all_fragments()

Every fragment cg knows, keyed by slug.

Built fresh rather than cached: the language registry is itself lazily discovered, and a stale copy here would be a second source of truth for something that already has one.

Source code in codingame_tools/language/toolchain/registry.py

def all_fragments() -> dict[str, CgToolchainFragment]:
    """Every fragment cg knows, keyed by slug.

       Built fresh rather than cached: the language registry is itself lazily discovered, and a
       stale copy here would be a second source of truth for something that already has one."""
    # Imported here, not at module scope: the language registry imports every plugin, each of which
    # imports `base`, which imports this package. At module scope that is a cycle -- the same one
    # `codingame_tools.language.vscode` resolves the same way.
    from ..registry import get_language, list_language_cg_ids

    table: dict[str, CgToolchainFragment] = {f.slug: f for f in SUBSYSTEMS}
    for cg_id in list_language_cg_ids():
        fragment = get_language(cg_id).toolchain_fragment
        if fragment is None:
            continue
        existing = table.get(fragment.slug)
        if existing is not None and existing != fragment:
            raise CgToolchainError(
                    f"two different toolchain fragments claim the slug {fragment.slug!r} "
                    f"({cg_id} collides with an existing definition)")
        table[fragment.slug] = fragment
    return table

default_languages

default_languages()

Every language cg can put in an image -- the default contents of the toolchain.

Derived, never a hardcoded list. A language is in the default set exactly when it declares a toolchain_fragment, so adding one is a single-module change and the two can never drift apart.

The default is everything rather than a minimal subset because the whole set costs about 1.9 GB: the languages that dominate (JDK, .NET, Node) share one Debian base instead of each dragging its own, so trimming saves far less than the confusion of having to choose. Subset builds remain available for anyone who wants one -- see CgSettingsData.toolchain_languages -- they are just not something a user should have to think about.

Source code in codingame_tools/language/toolchain/registry.py

def default_languages() -> list[str]:
    """Every language cg can put in an image -- the default contents of the toolchain.

       **Derived, never a hardcoded list.** A language is in the default set exactly when it declares
       a `toolchain_fragment`, so adding one is a single-module change and the two can never drift
       apart.

       The default is *everything* rather than a minimal subset because the whole set costs about
       1.9 GB: the languages that dominate (JDK, .NET, Node) share one Debian base instead of each
       dragging its own, so trimming saves far less than the confusion of having to choose. Subset
       builds remain available for anyone who wants one -- see `CgSettingsData.toolchain_languages`
       -- they are just not something a user should have to think about."""
    from ..registry import get_language, list_language_cg_ids

    return [
        cg_id for cg_id in list_language_cg_ids()
        if get_language(cg_id).toolchain_fragment is not None
    ]

fragments_for_languages

fragments_for_languages(languages)

Everything needed to build an image for languages, in install order.

The whole pipeline in one call: names to slugs, slugs to fragments, dependencies pulled in and ordered deterministically.

Source code in codingame_tools/language/toolchain/registry.py

def fragments_for_languages(languages: list[str]) -> list[CgToolchainFragment]:
    """Everything needed to build an image for `languages`, in install order.

       The whole pipeline in one call: names to slugs, slugs to fragments, dependencies pulled in and
       ordered deterministically."""
    return resolve_fragments(resolve_language_slugs(languages), all_fragments())

resolve_language_slugs

resolve_language_slugs(languages)

Fragment slugs for CodinGame language names, e.g. ["C++"] -> ["cpp"].

Raises:

• CgToolchainError –

  if a name isn't a known language, or is one with no container support -- distinguished, because "you typed it wrong" and "cg can't containerize that yet" need different fixes.

Source code in codingame_tools/language/toolchain/registry.py

def resolve_language_slugs(languages: list[str]) -> list[str]:
    """Fragment slugs for CodinGame language names, e.g. `["C++"] -> ["cpp"]`.

    Raises:
        CgToolchainError: if a name isn't a known language, or is one with no container support --
                           distinguished, because "you typed it wrong" and "cg can't containerize
                           that yet" need different fixes.
    """
    # Imported here, not at module scope: the language registry imports every plugin, each of which
    # imports `base`, which imports this package. At module scope that is a cycle -- the same one
    # `codingame_tools.language.vscode` resolves the same way.
    from ..registry import get_language, list_language_cg_ids

    known = {cg_id.casefold(): cg_id for cg_id in list_language_cg_ids()}
    slugs: list[str] = []
    for name in languages:
        cg_id = known.get(name.casefold())
        if cg_id is None:
            raise CgToolchainError(
                    f"unknown language {name!r}. Known: {', '.join(sorted(known.values()))}")
        fragment = get_language(cg_id).toolchain_fragment
        if fragment is None:
            raise CgToolchainError(
                    f"{cg_id} has no toolchain fragment yet, so it can't be built into an image. "
                    "Languages gain one as they gain a real build/run backend.")
        slugs.append(fragment.slug)
    return slugs