chore!: send modules archive over the proto messages by Emyrk · Pull Request #21398 · coder/coder

Emyrk · 2025-12-29T14:32:45Z

What this does

Dynamic parameters caches the ./terraform/modules directory for parameter usage. What this PR does is send over this archive to the provisioner when building workspaces.

This allow terraform to skip downloading modules from their registries, a step that takes seconds.

Wire protocol

The wire protocol reuses the same mechanism used to download the modules provisoner -> coder. It splits up large archives into multiple protobuf messages so larger archives can be sent under the message size limit.

🚨 Behavior Change (Breaking Change) 🚨

Before this PR modules were downloaded on every workspace build. This means unpinned modules always fetched the latest version

After this PR modules are cached at template import time, and their versions are effectively pinned for all subsequent workspace builds.

Emyrk · 2025-12-29T14:33:01Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

johnstcn

This does make template versions lock a specific set of module versions. Which is a behavior change for the better.

So if I understand correctly, this means that a template version that references one or more modules will always refer to the same versions of those modules that were resolved at template import time? This seems like a pretty big behavioural change that merits being called out in the release notes.

johnstcn · 2026-01-09T08:56:40Z

coderd/provisionerdserver/provisionerdserver.go

+		// This check is not perfect. If these conditions are not true, then the file is not a modules file.
+		if file.CreatedBy != uuid.Nil || file.Mimetype != tarMimeType {
+			return fail(xerrors.Errorf("file %s is not a modules file", fid))
+		}


Is there a particular set of files for which we could check the presence that would indicate that it is, in fact, a modules archive?

Yes, but that would require reading over the archive, which is however large. I do not think it's worth it.

Ideally maybe we could have some mimetype for our archives that indicate what kind of archive it is?

type/subtype;parameter=value

We could maybe do like application/x-tar;coder_type=modules or something?

Oh nice; I forgot about the parameter values you could stuff into mimetypes. That could work nicely.

If I do that, then it would not be backwards compatible. So all previous templates would not use this, since the mime types would be wrong

We also do not parse mime types right now, we just do direct string comparisons:

coder/coderd/files/cache.go

Lines 306 to 316 in e54f92b

var files fs.FS

switch file.Mimetype {

case "application/zip", "application/x-zip-compressed":

files, err = archivefs.FromZipReader(bytes.NewReader(file.Data), int64(len(file.Data)))

if err != nil {

return CacheEntryValue{}, xerrors.Errorf("failed to read zip file: %w", err)

}

default:

// Assume '"application/x-tar"' as the default mimetype.

files = archivefs.FromTarReader(bytes.NewBuffer(file.Data))

}

I do think it is a good idea, I just think we need to fix mime type handling in our codebase

That's fair, would be a good follow-up for later.

coderd/provisionerdserver/provisionerdserver.go

provisionersdk/tfpath/tfpath.go

provisionersdk/dataupload.go

provisionerd/runner/init.go

provisionerd/provisionerd.go

johnstcn · 2026-01-09T10:18:44Z

provisionerd/runner/runner.go


+	var cachedModulesTar []byte
+	// Download modules if cached in coderd
+	if r.job.GetWorkspaceBuild().Metadata.TemplateVersionModulesFile != "" {


(Not necessary in this PR, potential followup) I'm nervous about this unconditional calling of r.job.GetWorkspaceBuild() that potentially returns nil. Can we add a top-level check e.g.

jobWorkspaceBuild := r.job.GetWorkspaceBuild() if jobWorkspaceBuild == nil { // whoopsiedoodle }

That is a good point. Protobuf handling these union types is unfortunate.

Emyrk · 2026-01-09T14:24:58Z

So if I understand correctly, this means that a template version that references one or more modules will always refer to the same versions of those modules that were resolved at template import time? This seems like a pretty big behavioural change that merits being called out in the release notes.

I will mark this a breaking change and mention that 👍.

You do understand correctly. This is a positive change, because without this, a template can break from module changes.

johnstcn · 2026-01-09T17:05:54Z

coderd/provisionerdserver/provisionerdserver.go

+		versionModulesFile := ""
+		tfvals, err := s.Database.GetTemplateVersionTerraformValues(ctx, templateVersion.ID)
+		if err != nil && !xerrors.Is(err, sql.ErrNoRows) {
+			// Older templates (before dynamic parameters) will not have cached module files.


There shouldn't be any left though, right?

There are old template versions. They will be around forever.

Emyrk · 2026-01-09T17:12:58Z

Merge activity

Jan 9, 5:12 PM UTC: A user started a stack merge that includes this pull request via Graphite.
Jan 9, 5:25 PM UTC: Graphite rebased this pull request as part of a merge.
Jan 9, 5:33 PM UTC: @Emyrk merged this pull request with Graphite.

Experiment is no longer required, the new method will be released without an experiment and without a toggle Main PR is: #21398

**This is just the protobuf changes for the PR #21398 Moved `UploadFileRequest` from `provisionerd.proto` -> `provisioner.proto`. Renamed to `FileUpload` because it is now bi-directional. This **is backwards compatible**. I tested it to confirm the payloads are identical. Types were just renamed and moved around. ```golang func TestTypeUpgrade(t *testing.T) { t.Parallel() x := &proto2.UploadFileRequest{ Type: &proto2.UploadFileRequest_ChunkPiece{ ChunkPiece: &proto.ChunkPiece{ Data: []byte("Hello World!"), FullDataHash: []byte("Foobar"), PieceIndex: 42, }, }, } data, err := protobuf.Marshal(x) require.NoError(t, err) // Exactly the same output // EhgKDEhlbGxvIFdvcmxkIRIGRm9vYmFyGCo= on `main` // EhgKDEhlbGxvIFdvcmxkIRIGRm9vYmFyGCo= on this branch fmt.Println(base64.StdEncoding.EncodeToString(data)) } ``` # What this does This allows provisioner daemons to download files from `coderd`'s `files` table. This is used to send over cached module files and prevent the need of downloading these modules on each workspace build.

Serializing modules to disk prevents an external download

github-actions bot assigned Emyrk Dec 29, 2025

Emyrk mentioned this pull request Dec 29, 2025

chore: remove experiment "terraform-directory-reuse" #21397

Merged

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch 2 times, most recently from 3a1cf4a to f68e058 Compare December 29, 2025 15:29

Emyrk force-pushed the stevenmasley/remove_exp branch from 301568a to dea6d25 Compare December 30, 2025 14:43

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from d7c2bf0 to add8b5a Compare December 30, 2025 14:43

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from d35a9f1 to eae738d Compare January 6, 2026 16:16

Emyrk force-pushed the stevenmasley/remove_exp branch from fe1156c to 408096b Compare January 7, 2026 14:37

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from eae738d to b180036 Compare January 7, 2026 14:37

Emyrk changed the base branch from stevenmasley/remove_exp to graphite-base/21398 January 7, 2026 14:49

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from b180036 to f65837d Compare January 7, 2026 14:49

Emyrk changed the base branch from graphite-base/21398 to 01-07-chore_update_protobuf_to_reuse_file_request January 7, 2026 14:49

Emyrk mentioned this pull request Jan 7, 2026

chore: update protobuf to reuse file request #21447

Merged

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from f65837d to b54a67a Compare January 7, 2026 14:51

Emyrk force-pushed the 01-07-chore_update_protobuf_to_reuse_file_request branch 2 times, most recently from 5d2e3d6 to a3c50b5 Compare January 7, 2026 15:02

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from b54a67a to 9ea352f Compare January 7, 2026 15:02

Emyrk force-pushed the 01-07-chore_update_protobuf_to_reuse_file_request branch from a3c50b5 to 1d3e35e Compare January 7, 2026 16:15

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from 9ea352f to 035f01e Compare January 7, 2026 16:15

Emyrk force-pushed the 01-07-chore_update_protobuf_to_reuse_file_request branch from 1d3e35e to 83c73d9 Compare January 7, 2026 16:48

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch 2 times, most recently from 7529e06 to 8e1a315 Compare January 7, 2026 17:24

Emyrk force-pushed the 01-07-chore_update_protobuf_to_reuse_file_request branch from 83c73d9 to 6e60c5f Compare January 7, 2026 17:24

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from 8e1a315 to 5e7de71 Compare January 7, 2026 17:27

Emyrk force-pushed the 01-07-chore_update_protobuf_to_reuse_file_request branch 2 times, most recently from ecac30d to 2df9ca9 Compare January 7, 2026 17:44

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch 3 times, most recently from f708df5 to 8af46fb Compare January 7, 2026 17:51

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from 9344fd5 to dc40539 Compare January 8, 2026 18:07

Emyrk force-pushed the 01-07-chore_update_protobuf_to_reuse_file_request branch from 259ab00 to 19a5a3b Compare January 8, 2026 18:07

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from dc40539 to 16d9aac Compare January 8, 2026 18:11

Emyrk force-pushed the 01-07-chore_update_protobuf_to_reuse_file_request branch 2 times, most recently from c2c8425 to 5b0df75 Compare January 8, 2026 19:17

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch 2 times, most recently from 1aaa81b to 715e5a1 Compare January 8, 2026 19:33

Emyrk force-pushed the 01-07-chore_update_protobuf_to_reuse_file_request branch from 5b0df75 to ecf4457 Compare January 8, 2026 19:33

Emyrk marked this pull request as ready for review January 8, 2026 19:39

Emyrk requested a review from johnstcn January 8, 2026 19:40

johnstcn reviewed Jan 9, 2026

View reviewed changes

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from 715e5a1 to e54f92b Compare January 9, 2026 14:23

Emyrk force-pushed the 01-07-chore_update_protobuf_to_reuse_file_request branch from ecf4457 to 9f5aab4 Compare January 9, 2026 14:23

Emyrk added the release/breaking This label is applied to PRs to detect breaking changes as part of the release process label Jan 9, 2026

Emyrk changed the title ~~chore: send modules archive over the proto messages~~ chore!: send modules archive over the proto messages Jan 9, 2026

Emyrk requested a review from johnstcn January 9, 2026 16:15

johnstcn reviewed Jan 9, 2026

View reviewed changes

johnstcn approved these changes Jan 9, 2026

View reviewed changes

Emyrk added a commit that referenced this pull request Jan 9, 2026

chore: remove experiment "terraform-directory-reuse" (#21397)

89f4d60

Experiment is no longer required, the new method will be released without an experiment and without a toggle Main PR is: #21398

Emyrk changed the base branch from 01-07-chore_update_protobuf_to_reuse_file_request to graphite-base/21398 January 9, 2026 17:14

Emyrk changed the base branch from graphite-base/21398 to main January 9, 2026 17:23

Emyrk requested a review from spikecurtis as a code owner January 9, 2026 17:23

chore: send modules archive over the proto messages

ced4ebd

Serializing modules to disk prevents an external download

Emyrk force-pushed the stevenmasley/modules_over_the_wire branch from e54f92b to ced4ebd Compare January 9, 2026 17:24

Emyrk merged commit 60b3fd0 into main Jan 9, 2026
31 checks passed

Emyrk deleted the stevenmasley/modules_over_the_wire branch January 9, 2026 17:33

github-actions bot locked and limited conversation to collaborators Jan 9, 2026

	var files fs.FS
	switch file.Mimetype {
	case "application/zip", "application/x-zip-compressed":
	files, err = archivefs.FromZipReader(bytes.NewReader(file.Data), int64(len(file.Data)))
	if err != nil {
	return CacheEntryValue{}, xerrors.Errorf("failed to read zip file: %w", err)
	}
	default:
	// Assume '"application/x-tar"' as the default mimetype.
	files = archivefs.FromTarReader(bytes.NewBuffer(file.Data))
	}

Conversation

Emyrk commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this does

Wire protocol

🚨 Behavior Change (Breaking Change) 🚨

Uh oh!

Emyrk commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

johnstcn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Emyrk commented Jan 9, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Emyrk commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge activity

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Emyrk commented Dec 29, 2025 •

edited

Loading

Emyrk commented Dec 29, 2025 •

edited

Loading

Emyrk commented Jan 9, 2026 •

edited

Loading