Skip to main content

Exporting & importing

So far we've talked about pushing YAML to an environment. Just as often, you'll want to pull existing configuration out of one — to bring a customer's existing setup under version control, to copy a resource between environments, or simply to see how something is built. That round-trip — export → edit → apply — is one of the most useful patterns in cotctl.

This page uses surveys as the worked example, because they have the richest export options. The same list / get / export / apply shape repeats across the other entity groups: roles, property-types, properties, users, jobtitles, workflows, and the newer bots, bot-types, routines, schedules and slas.

Not every group has all four verbs. The shape is a pattern, not a guarantee — a few groups differ:

  • bot-types is read-only. It exposes only list and versions <BotType> (the live catalog of ParametrizedBot types). There's no bot-types apply; you author bots with cotctl bots.
  • slas need their state machine. slas get, slas export and the by-code paths require --state-machine <smCode>, because there's no global by-code lookup for SLAs.
  • Some groups add verbs. routines has test <code> (run a routine immediately — real side effects); schedules has activate / deactivate / logs.

Naming note. The old cotctl get surveys and cotctl export survey commands were removed. Use the entity-scoped forms — cotctl surveys list, cotctl surveys export, and so on.

Finding what's there: list​

Before exporting, you usually need to find the resource. list shows what exists, with search and paging:

cotctl surveys list -c acme
ID                         NAME                  CODE             VER   ACTIVE   MODIFIED
-------------------------------------------------------------------------------------------
507f1f77bcf86cd799439011 Order Request Form order_request 3 true 2024-03-15
507f1f77bcf86cd799439012 Approval Survey approval_srv 1 true 2024-02-20

Useful options: -s/--search <text> to filter, --all to include inactive resources, -l/--limit and -p/--page for paging, and --json for machine-readable output.

What --search matches depends on the resource — <resource> list --help describes it — and bot-types and slas have no --search at all. surveys list --search matches the name only — to find a survey by its code, use surveys list --code <code>, an exact lookup.

Looking at one: get​

To inspect a single resource without writing a file:

cotctl surveys get order_request -c acme

Add --populate to include the full question list (output switches to YAML automatically, since a table can't show nested questions), or -o json to get JSON.

Pulling it out: export​

export is what brings a resource down as a YAML file you can version and re-apply:

cotctl surveys export order_request -c acme -o ./order_request.yaml

-o is a path, not a format. A common first mistake is -o yaml. The -o/--output flag is the file path to write to; use --format to choose the format. Passing a format keyword to -o is a hard error with a message telling you exactly this — for -o json or -o yaml, that the export is always YAML.

Two export formats are available, and since 0.14.0 --format takes only these two values, in lower case — anything else (json, yaml, RAW) exits 1 before anything is read, where it used to export the simplified format silently:

--formatDescription
simplified (default)Human-readable, with a clean questions[] array — what you want for version control
rawThe raw API representation — and the supported escape hatch for a survey the simplified format cannot express
# Default simplified YAML, printed to stdout
cotctl surveys export order_request -c acme

# Raw format, written to a file
cotctl surveys export order_request -c acme --format raw -o ./order_request_raw.yaml

--format raw is not just for debugging. The simplified format is a model of a survey, and an older or hand-built survey does not always fit it. When it does not, export refuses and exits 2, naming the question that cannot be expressed and pointing you here. --format raw exports that survey as stored, with two clean-ups so validate accepts the file: a reset identifier that names no question of the export is left out with a warning, and the empty [""] reset the webclient's builder stores is written as []. apply takes it back, so it is the answer, not a workaround. Expect one update on the way back: the first apply of a raw export over a survey built in the webclient usually still sends one (Would UPDATE in a dry run), because what the webclient stored differs from what a raw apply writes. From the second apply on, the raw round trip changes nothing.

A script wrapping surveys export can branch on exit 2 and retry with --format raw without reading the message — it has exactly one meaning in this command. See CI/CD. In 0.12.0 this refusal changed from exit 1 to exit 2.

A simplified export can also succeed and leave something out, which is the worse case: the YAML looks fine, gets committed, and the next apply is what removes the missing questions. Since 0.12.0 cotctl warns on stderr whenever it drops content — an unexpected chat bubble, a legacy bubble packing several questions into one, or a standalone text question absorbed as another question's label. Read stderr on an export you are about to commit. stdout stays clean, so piping and -o are unaffected.

Since 0.14.0 the export also folds a title named labelQuestion<identifier> (as some solution presets name them) into its question, instead of exporting it as a text question and leaving the real question out — and the dry run of a survey update flags the questions it would deactivate as ⚠ DANGER. An export made with 0.13.0 or earlier can carry exactly that gap: re-export before re-applying it.

Since 0.15.0 any two-entry chat bubble — a text followed by a question that takes a title — is read as that title and its question, whatever the text is called (kilometrajeLabel, a timestamp, …), and an apply rebuilds the stored title in place, keeping its identifier and its _id. A title's own condition, help text and required are not kept by the simplified format: the export warns about each and points to --format raw, and the dry run warns about the condition.

Three more cases now warn instead of failing silently or failing late:

  • A chat bubble whose questions were all deactivated or deleted is skipped with a SKIPPED warning, and the export exits 0 (it used to exit 2). The warning names the SurveyChat an apply of the export deactivates.
  • A condition operator the simplified format does not accept, such as the = older webclients stored, is written as stored with a warning pointing to --format raw — validate and apply still refuse that file.
  • A reset identifier that names no question of the export is left out with a warning, in both formats. The raw format also writes the empty [""] reset the webclient's builder stores as [], so validate accepts the file it just exported.

Since 0.16.0, a few more surveys export to a file their own validate accepts:

  • A raw exec.onPlay.button may lack any of label, type, theme and debounceTime, or be null, as the server allows. A button saved without debounceTime used to fail the raw export's own validate.
  • A chat bubble of a type the update does not accept back, such as text/system, is skipped by --format raw with a warning, as the simplified format already did: applying the export deactivates that SurveyChat, and the dry run says so in a warn line that --fail-on-destructive lets through. The simplified export no longer points to --format raw for it.
  • A jobTitle filter stored as "*" or as a single code is exported: "all job titles" as jobs: "*", and a legacy single code as a one-code list, which the apply writes back as stored. A stored list holding "*" exports as jobs: "*", without the list's other codes, with a warning that the next apply writes it back as the string. A stored value that is neither is named in a warning.
  • The simplified format keeps a question's exec.onChange, exec.presave and exec.filter. It used to leave them out, so re-applying a simplified export erased them from every question and table column that stored them. Both formats now export and send them, with a warning that the webclient never runs them.

Exports write only what is stored​

Since 0.14.0, an export writes the keys the resource actually stores and nothing else. Earlier versions filled in, with its default, every key the resource lacked — and re-applying that export sent those defaults as changes. Now re-applying an unchanged export sends nothing, which matters most for a running schedule, whose cron any update stops.

Two consequences to plan for:

  • A script that reads keys from an exported YAML has to accept their absence — isActive, accessRoles, extraData, a schedule's cronTimeZone or timeoutMinutes, and so on. Read an absent key as the default the export used to write.
  • Re-export a YAML exported with 0.13.0 or earlier before re-applying it. Those files carry defaults the resource may not store. The costly case is a schedule stored without cronTimeZone: the old export wrote America/Santiago, so re-applying it moves the cron from the scheduler's own zone to Santiago — three or four hours away — and the isActive: true it wrote relaunches it there right away. apply warns when an update sets a zone on such a cron.

properties export writes Property references by code (0.16.0). In a COTProperty node of schemaInstance it writes the code of the Property referenced, where it used to write its _id, so the export round-trips by code. An _id it cannot name — a deleted or deactivated Property's — stays, with a warning, and so does every reference under a node whose PropertyType is not active or cannot be found (an inactive one included, when the profile lacks admin-properties-read). Apply such an export with 0.16.0 or later: an older cotctl sends the code as written and the backend refuses it. An export taken before 0.16.0 still applies, and a script that reads ids out of an export now reads codes.

workflows export writes taskPaginationLimit and defaultSortBy when the Workflow stores them (0.16.0); see Workflows.

A few exports still send an update on their first re-apply: users export leaves out an inactive access role (with a warning), so the re-apply removes it from the user; routines export writes the routine's code as its display when it stores none; the first re-apply of a survey built in the web app rewrites it in cotctl's shape — in the raw format too; and the first re-apply of a survey that stores the legacy subtype: long_text sends short_text in its place.

Keeping scripts out of YAML​

Surveys can carry inline JavaScript (exec hooks). For cleaner version control, --extract-scripts <dir> pulls those scripts out into separate files and replaces them with file:// references in the YAML:

cotctl surveys export order_request -c acme \
-o ./order_request.yaml \
--extract-scripts ./scripts/

Now the JavaScript lives in real .js files your editor and Git can handle properly.

The round-trip in full​

Putting it together, the export–edit–apply loop looks like this:

# 1. Export the live resource
cotctl surveys export order_request -c acme -o order_request.yaml

# 2. Edit order_request.yaml in your editor, commit it to Git

# 3. Validate, then apply the change back
cotctl validate -f order_request.yaml
cotctl apply -f order_request.yaml -c acme

This is also how you promote between environments — export from staging, apply to production (with the matching -c profile). One check behaves differently there: the required bot data entries are compared with the stages stored in the target company, so a stage that doesn't exist there yet is new, and every required entry it lacks is refused — even when the same export re-applies cleanly where it came from.

Deactivating instead of deleting​

Cotalker favors deactivation over hard deletion. To take a survey out of use without losing it:

cotctl surveys deactivate order_request -c acme

It sets isActive: false after a confirmation prompt (-y skips the prompt in scripts).

See also​