Ask Spark

Ask Spark is the assistant in the engineering workspace. Open Ask Spark from the Designer, Projects or Gateway Settings header, or press Alt+A outside a text editor. The same conversation follows you between these pages. It works with the signed-in user's permissions and the context chips shown above the message box.

Configure Gemini

1. Sign in as a gateway administrator and open Gateway Settings → AI.
2. Enter a Gemini API key. The gateway encrypts it with its data-protection keys; saved keys are never returned to the browser. Leave the model at gemini-3.8-flash, or enter another compatible Gemini model ID.
3. Enable Ask Spark, save, then test the configuration. Provider errors appear in this panel. A key is required for chat, image understanding and voice transcription.
4. The parallel read limit defaults to 4. Independent read calls can run together. Draft edits, saves and other mutations run in order.
5. Model steps per message defaults to 100 and accepts 1–1,000. A model response counts as one step, including a discovery request or several parallel tool calls. After the configured number of steps, the gateway requests one final answer with tools disabled. Saving a new limit applies to new messages; a running message keeps its starting limit.
6. Optionally set Monthly AI token allowance. The default 0 is unlimited. This allowance covers all users and projects on this gateway, including chat, transcription and connection tests. Refresh usage shows the current UTC month's counters. This is a token allowance, not a currency budget.
7. Log raw Gemini requests and responses is enabled by default. Clear it and save to stop recording new provider traffic. Each request and reply is saved in a timestamped text file under <gateway data directory>/askspark/; the local development gateway uses .data/development/askspark/.

The gateway sends the conversation, attached images/audio, visible context and requested tool results to Google's Gemini service. Review your organization's data handling requirements before entering production information. Connection passwords and API keys use separate secure dialogs and are excluded from tool results. Do not paste credentials into chat, screenshots or authored code. Chat history is private to the authenticated user and encrypted on the gateway. Each user retains up to 20 conversations within a 64 MiB storage budget; older conversations are removed when that limit is reached. Use the history panel to delete a conversation explicitly. Configuration backups include encrypted AI settings and their protection keys; they exclude conversations, recordings and the provider log.

Provider log

The provider log records the actual outbound request and inbound response bodies without additional filtering, including model text, tool calls/results, source code, image/audio base64 and provider error responses. It also records context-cache creation/deletion and token-counting traffic, so cached prompts can be inspected. HTTP authentication headers are never recorded. This is a separate plaintext gateway diagnostic log, shared across users of this gateway, rather than the encrypted per-user chat history. Deleting a conversation does not erase its provider log entries.

The gateway restricts the log directory and its files to the gateway service identity, Administrators and SYSTEM on Windows. Unix permissions are 0700 for the directory and 0600 for files. On the first enabled write, it also secures existing exchange files without changing their contents. These filesystem restrictions do not encrypt the logs or change the default logging setting or retention policy.

Each Gemini HTTP request and its reply share one file named <UTC timestamp>-<exchange ID>.txt, for example 20261003T143202.9994836Z-e887bb6b09524d39b58fdc97a77e3e6f.txt. One chat message can create several files as the model calls tools across rounds. Token counting and context-cache operations also get their own files. Records include the operation, timestamp, exchange ID, direction and body byte count, followed by the exact body text.

Files have no logging size cap, truncation or automatic deletion. Requests larger than 10 MB are preserved in full, and later exchanges do not overwrite earlier files. Normal provider request/response size limits still apply: interrupted or oversized responses are marked incomplete and contain the bytes received within those limits. A request without a reply can indicate an interrupted process. Disabling logging stops new entries and leaves existing files in place. The former ask-spark-provider.log is preserved unchanged. Log write failures do not fail the assistant's request.

Talk, type or paste a screenshot

Use the microphone beside Ask Spark or inside the chat. Grant browser microphone access, speak, then stop recording. The returned transcript goes into the message box: edit it and press Send when ready. Cancel discards the recording. Microphone capture requires a secure browser context such as HTTPS or localhost; recording/transcription failures leave the text composer available.

Paste a screenshot into the composer, or use its image attachment button. Review the thumbnails and remove anything unnecessary before sending. PNG, JPEG and WebP images are supported, with up to four images and bounded request size. Images are sent only when you send the message. A screenshot helps explain visual intent; editable project information comes from the designer tools. The assistant must treat text in screenshots and tool results as content, not new instructions.

Ask Spark's answers render Markdown: headings, bold and italic text, lists, block quotes, links, tables, inline code and fenced code blocks. Use Copy code to copy a code block. Wide tables and code scroll within the chat. Your messages remain plain text. Embedded HTML is omitted, and Markdown images appear as links instead of loading remote content automatically.

Ask “Open the Ask Spark workshop project and its workshop screen.” The assistant can open a permitted project in the Designer, wait for it to load, and continue the same conversation with the new project context. It can also switch to an existing screen or template. Unsaved project, query, script or connection edits block navigation away from that work. Other navigation tools provide links to settings sections.

Designer workflow

1. Open a screen and select the relevant components. Context chips identify the project, document and selection. Pin context to keep it while inspecting something else; remove chips to narrow what is shared.
2. Ask, for example, “Align the selected controls on their left edges and make them the same width.” Ask Spark inspects the current draft and uses the existing layout operations.
3. Review the action receipts. Draft changes create the same Undo checkpoints as manual edits. The chat's Undo button applies only while that exact result remains current; it refuses to overwrite subsequent edits.
4. Ask “Bind this value to [default]Line1/Speed”, “Explain this button's script”, or “Show missing references on this screen.” Tools can inspect and edit property expressions, named-query bindings, component events, JavaScript/Python code, custom properties, template parameters, messages and input state.
5. Ask “Validate the draft and open read-only preview.” Validation uses gateway authoring rules. Diagnostic snapshots evaluate current bindings and identify missing references; they do not execute arbitrary scripts, database updates or equipment actions.
6. Save the project draft when ready. Publication remains a separate reviewed action using the existing publication token and revision checks.

Ask Spark can make several edits in one model round. The spark_designer_apply_edits tool validates an entire batch before committing it as one Undo checkpoint. Consecutive draft-edit tools can also build on each other within the same response. Every draft mutation identifies its snapshot; the browser permits only its own immediately preceding edit to advance that snapshot. Manual changes, intervening operations and stale snapshots still stop the edit. Screen elements keep their saved Z order unless an explicit arrangement tool changes it. Names and IDs remain distinct; deleting a referenced resource is blocked by the same dependency checks used in the editor.

The component-schema tool returns a compact reference for the requested type, retaining its complete runtime property constraints and basic action guidance. The assistant can request specific additional contracts with include, such as actions, scripts, componentEvents or tableEdit, or use detail:full for the complete reference. This avoids repeating the full scripting and event catalogs for every component type. Button action is a string such as script; that action's script field contains gateway Python. For example, result = {'message': 'Lunch selected.'} produces button feedback when the action runs with its usual permissions. Component message actions use messageType, scope and payload to dispatch application messages; they are separate from button feedback. Input change/commit events and component interaction events have their own supported names and JavaScript/Python contracts.

For a shared change, the assistant can update many component IDs with one property patch; unchanged scripts and other properties are retained. Different dependent edits use the atomic batch tool. The assistant is guided to reuse inspected assets, avoid disposable write probes, and capture at useful visual milestones. Its completion summary should distinguish an unsaved draft, saved changes and publication, and distinguish structural validation from visual inspection and actual interaction testing.

Opening Designer Preview keeps Ask Spark available so it can finish its answer. While Preview is open, assistant tools are limited to reads; leave Preview before further draft or gateway configuration changes. Preview runtime requests retain their existing capability and live-action restrictions. Published operator tools continue to target the published project, not the unsaved draft.

When the assistant saves or deletes named queries or scripts, the Designer refreshes those resources before the next tool runs. Clean query and script editors update immediately. An editor containing unsaved text retains that text and shows a notice about the gateway change. New image assets also refresh automatically. Tag changes that require review follow the preview-then-apply workflow: the assistant first obtains the proposal and token, then applies that exact proposal after any required approval.

Check the rendered layout

Ask “Capture this canvas and check whether the labels overlap.” The capture tool returns an image of the current rendered Designer screen or template to Gemini, together with the document ID, snapshot token and dimensions. Small captures stay PNG; larger captures use high-quality WebP when it is smaller, preserving transparency and dimensions before any size reduction. The frame fits within 2,048 pixels per side and 5 MiB, so large screens may be reduced. It hides selection handles, the editor grid and private/password fields. External images or fonts must first be imported into the gateway; inaccessible resources cause an explicit capture error. A capture checks the visible layout. It does not execute operator actions or prove that a published runtime works.

The conversation keeps the latest successful canvas capture's pixels. Older capture receipts retain their document, snapshot and dimensions, with an explicit superseded-image notice. A failed capture does not remove the last successful image. Original pasted references remain available; ask for a new capture to inspect an older view again. This bounds repeated canvas-image traffic without removing tool-call pairs or the model's signed response parts. Raw provider files retain each request and response as actually sent or received, including captures present in those requests.

Turn a pasted screenshot into image assets

1. Paste or attach your screenshot and send it. Ask Spark receives its image ID and original dimensions.
2. Ask “Crop the three tiles into separate project assets named Speed tile, Temperature tile and Count tile.” The assistant proposes names and bounding boxes in the original image's pixels.
3. Review the crop thumbnails, dimensions and exact arguments. One approval covers the batch of up to 16 crops.
4. After approval, the browser crops the retained image and uploads each image through the normal project asset endpoint. The assets become available in the Designer. Ask Spark can then place image components using the returned asset IDs.

Crops that fit the 512 KiB asset limit remain PNGs at their original resolution. Larger crops are compressed as WebP at the original dimensions first, then reduced proportionally only if necessary. Transparency and the selected area are preserved; cropping does not enlarge or improve the source resolution. A 130 × 90 pixel tile can look soft when stretched. The receipt includes each asset's actual dimensions, format and size. All bounds and names are checked and all crops are prepared before uploads start. Identical image content reuses the existing asset and its saved name; the receipt includes both the requested name and actual asset metadata. Uploads run in order and stop on a failure or cancellation; the receipt lists already-created assets and any unattempted crops. Approval does not make separate asset uploads transactional. Inspect that receipt before retrying.

Sent images remain in memory in the active browser conversation, up to 12 images or 32 MiB of encoded data. They are not stored in browser local/session storage. Opening another conversation, starting a new one, signing out or reloading releases those originals; paste the image again to crop it afterward. Crops use only the images you supplied, never a model-provided download URL. Providing a new image or entering a password still uses the attachment control or separate secure prompt.

On wide desktops the assistant docks beside the inspector. At intermediate widths Properties / Ask Spark switches the right pane while retaining inspector state. On narrow screens the assistant opens as a sheet. Closing it preserves the conversation and draft. The assistant never changes the saved screen size to make space.

Gateway workflow

Open the appropriate settings section and describe the task. Examples:

• “Inspect my MQTT mappings and explain why no tags have appeared.”
• “Browse this OPC UA connection and prepare an import of the selected points.”
• “Create a named query for the work-orders table, with a station parameter.”
• “Review the backup schedule and show the latest result.”
• “Explain recent connection errors and bad-quality tags.”

Tools cover project lifecycle and packages; connections, drivers, certificates and browsing; tag models and imports; source mappings and diagnostics; named queries and scripts; alarms and history; deployment, backups and recovery review; users, project grants, sessions and audit. The shipped declarations in apps/web/src/askSparkGatewayTools.json and askSparkDesignerTools.json, plus the gateway's built-in find_tools and spark_open_project declarations, define the tool inventory.

Tool discovery and caching

Each model round starts with a small permitted core for navigation, project inspection, designer search and find_tools, plus the current page's tool category. The prompt also contains a directory of permitted tool names with short summaries. Full definitions for other categories are loaded only when needed. For example, while editing a screen, a request about backups can first call find_tools for backup tools; the matching full schemas are available from the next model round. Searching by an exact tool name returns only that definition. Discovered tools stay loaded in that conversation and are checked against current permissions before every round. Discovering a tool does not execute it or approve an operation.

If the model calls a permitted tool before loading its definition, the gateway returns a tool_not_loaded receipt to the model and continues discovery automatically. No calls from that mixed round execute, including otherwise loaded calls; no approval is consumed. The model must discover the missing definitions and submit fresh calls with valid arguments. Unknown tools and permission failures remain blocked. Recovery counts toward the configured step budget; after three unloaded-tool rounds without handing an executable round to the browser, the gateway requests a final answer without tools instead of continuing an expensive retry loop. Tool/context errors can be dismissed in chat; Refresh AI status is reserved for provider/service errors and does not itself make a Gemini test request.

The gateway also checks Gemini's completion status before accepting a response or executing its calls. A malformed function call, truncated generation or empty response can trigger up to two automatic repair attempts, within the message's configured step limit. No calls from a failed generation execute, even if part of the response looks usable. The repair asks for fresh structured calls and smaller batches of at most ten components. Earlier successful changes stay in the draft and are not replayed. This recovery leaves the initial tool set unchanged and preserves the usual discovery, permission and approval checks.

Every repair attempt counts toward the model-step limit and monthly usage. If three consecutive generations fail (the initial failed attempt plus up to two repairs), the step budget runs out, or the provider reports a non-retryable completion status, the chat explicitly reports that the task is incomplete. An accepted generation resets this consecutive-failure counter. Safety blocks and unknown provider failures do not trigger automatic retries. Raw provider files preserve the failed responses, including diagnostic text; malformed diagnostic source is not sent back to the model as executable calls or conversation content. Inspect the current draft and ask to continue the remaining work after an incomplete stop.

When a discovery result contains an exact copy of a currently loaded function definition, the provider request uses a reference to that declaration instead of repeating its description and parameters in conversation history. The stored discovery result remains complete, and definitions that are unavailable or have changed are preserved. Function-call IDs, response ordering and model signatures are unchanged.

Full tool descriptions are preserved. Tool results are bounded and include pagination or narrowing guidance rather than sending an unlimited resource inventory. Permissions, project revisions and existing approval requirements still apply after discovery. A model cannot call an undeclared tool merely because its name appears in the directory.

When supported by the selected Gemini model, the gateway caches the static system instructions, tool directory and loaded definitions on Gemini for one hour. It keeps at most eight cache entries per running gateway process, separated by authenticated context, project/page, model, settings revision and exact definitions. Messages, screenshots, recordings and returned application data are not placed in this explicit cache. Unused entries are evicted; remote entries remaining after a gateway restart expire according to their one-hour TTL. Models or prefixes that do not support caching fall back to ordinary requests. Final answers at the step limit use an uncached request without tools.

There is no per-message token allowance or aggregate tool-call cap that silently removes tools. The configured step limit bounds model rounds. Request size, context size, individual response size and per-round call bounds remain in force.

Monthly usage charges Gemini's reported total tokens minus cached input tokens. Cached tokens are displayed separately; output and thinking tokens count toward the allowance. A limited request first counts its full prompt and reserves that count plus its maximum response size, then reconciles to actual usage. This conservative admission requires sufficient headroom even for a cached request. Concurrent requests share the same durable ledger. Known rejected requests release their reservation; interrupted requests with an unknown provider outcome retain it across restarts. The allowance resets at the start of each UTC calendar month. This accounting is an application limit, not an invoice estimate: Google's cached-token and cache-storage charges still apply separately. Usage counters are local operational data and are not included in configuration backups.

The monthly panel also separates Input tokens (including cached input), Output tokens (generated responses, excluding thinking) and Thinking tokens. These counters persist across restarts and reset with the UTC month. Earlier usage records have only totals, so their breakdown shows an em dash until categorized usage is recorded. Historical or uncategorized tokens are identified explicitly; the category cards never invent an input/output split, and the original monthly totals and allowance accounting remain intact.

Consequential actions show a review card with the exact operation and arguments. Approval is tied to that pending call; the model cannot grant its own approval. Declining returns a refusal to the conversation. Existing API authorization, revision checks, dependency checks, publication reviews and backup recovery isolation still apply. A tool is unavailable when the user lacks its permission.

Changes made through gateway tools are saved gateway resources. Designer queries, scripts and assets refresh automatically as described above. Reload an already-open settings editor after other gateway changes; revision-aware editors reject stale saves. Designer draft tools operate on the open unsaved project instead. Use the draft Save tool before requesting publication. Code edits save code; running a script is a separate, permission-checked operation.

Test the published operator runtime

Ask “Check the operator session for this project.” Runtime tools are discoverable when the engineering account has Design permission and the corresponding View, Operate or Commands grant. They target the project's published runtime; Designer Preview remains the place to test an unpublished draft.

Runtime testing uses a separate operator session for the same signed-in account. If that session is missing, Ask Spark can request an approved operator sign-in, then present a secure password prompt. The username comes from the current engineering account. The password and operator session credentials never enter the conversation or Gemini request. If another account is already signed into the operator application, Ask Spark refuses to replace it. Each operation checks the project, required grant and expected account again on the gateway.

Once authorized, the assistant can inspect the published runtime and invoke the available runtime tools. Writes and commands retain their review cards and the same server validation as manual operator actions. A failed or interrupted mutation is not automatically retried, because the first request may already have completed. Publication, real equipment actions and script execution are separate operations; a successful visual capture does not authorize them.

Workshop

examples/ask-spark.json is an independently authored synthetic canvas with three misaligned readouts. Import it as a separate development project, configure AI, and open the designer. No industrial equipment, database, gateway tags or executable scripts are required.

1. Select the three readouts and ask for left alignment and equal widths. Confirm their Z order stays unchanged.
2. Use the action's Undo, then repeat with different spacing.
3. Paste a screenshot and ask for an explanation of the layout before requesting changes.
4. Dictate a request, stop recording, edit the transcript, then send it.
5. Ask for a reusable faceplate from the selected controls. Inspect the new template and its parameters.
6. Ask to validate, save and preview. Review publication separately if you want an operator version.
7. Ask “Explain these three readouts in a Markdown table, with a checklist underneath.” Verify the table and checklist render in the answer. Ask for a short code example and try its Copy code button.
8. Ask “Find the tools for reviewing backups; explain what each does without changing anything.” Ask Spark discovers that category without granting approval to use its write operations. Continue with a permitted read if desired.
9. In AI settings, verify the model-step default is 100. Temporarily choose a smaller limit in this development gateway, save, and start a new message. At the limit the assistant summarizes with tools disabled. Restore your preferred limit afterward.
10. From Projects or Gateway Settings, ask to open this workshop and its screen. Continue in the same conversation, ask for a batch of layout edits, then ask the assistant to capture and critique the rendered canvas.
11. Paste a synthetic screenshot you created and request two named crops. Review the thumbnails, approve once, and confirm the assets appear without reloading. Keep their native resolution when placing them on the screen.
12. Optionally create a harmless named query or script on a disposable gateway, then ask the assistant to update it. Confirm a clean Designer editor refreshes, while unsaved text is preserved with a notice. This optional exercise requires its normal gateway resources and permissions.
13. Ask for the same foreground color on all three readouts and request one shared update. Capture, change the spacing and capture again. With raw logging enabled, inspect the latest provider file: it should contain the latest canvas image, any original pasted references, and metadata for superseded captures. Ask for a focused component schema, then a specific event contract, to compare compact and on-demand detail. These optimizations reduce repeated context; actual billed tokens depend on the model and task.
14. On the synthetic canvas, ask for twelve clearly labeled overlay buttons in small batches and request a final capture. Confirm the actual button count and placement before accepting the completion summary. If the provider returns a malformed, truncated or empty generation, earlier successful edits should remain and recovery should continue with fresh calls; a repeated failure or exhausted step budget should produce an explicit incomplete result. Do not deliberately force provider failures or expect this live exercise to prove the retry path. With raw logging enabled, the failed response and each repair request remain separate timestamped exchanges.
15. After explicitly publishing this synthetic workshop, ask to check its operator session and read its published runtime. Use the same account's secure sign-in prompt if necessary. No equipment writes are needed for this exercise.

Requires Windows preview.13 or a later compatible build containing Ask Spark. Windows preview.12 and Docker preview.11-docker.1 release assets do not contain this feature. See the release ledger for exact-package verification and provider acceptance limits. A Gemini account/key and outbound access to generativelanguage.googleapis.com are required for live AI calls and may incur provider charges. The workshop itself remains usable manually without AI.

Implementation and verification

The gateway calls Gemini's fixed HTTPS generateContent endpoint using parametersJsonSchema declarations. It preserves raw model content and thought signatures across function-call continuations. The browser runs only registered tool handlers and returns all results for a model turn together. Tool IDs, continuation tokens, exact approval arguments, user ownership and budgets are checked on the server. No model-supplied URL, tool definition or arbitrary browser evaluator is used.

Canvas captures travel as ordinary image parts alongside their tool results. The gateway also adapts captures in saved conversations before sending them to Gemini. Provider completion status is checked before calls are authorized or a final answer is accepted. If automatic generation repair cannot finish after a successful edit, that edit remains in the Designer draft; send a follow-up in the same conversation to inspect the current state and continue.

The API key is never bundled into the frontend. Voice uses MediaRecorder and the gateway transcription endpoint, not browser speech recognition. Audio is released after transcription, except for its encoded request body in the provider files when logging is enabled. Image requests validate MIME/signatures and size bounds before provider submission. Conversations are encrypted and isolated by user; the optional provider log is plaintext and gateway-wide. The recovered-gateway quarantine blocks external AI calls until existing recovery checks permit them.

The normal build gates run frontend/backend lint, offline unit and acceptance tests, and cyclomatic complexity checks. Ask Spark tests cover tool coverage, permissions, discovery continuations, configured step limits, cache isolation and fallback, monthly admission and accounting, Markdown rendering, secret redaction, signed provider replay, approval replay/decline, concurrent reads and ordered edits, atomic edit batches, image/audio bounds, retained image crops, rendered canvas capture, authorized project transitions, operator audience and account isolation, resource refresh, stale snapshots and Undo. Fake-provider tests do not establish live Gemini availability; the AI settings test verifies the configured account/model.

Provider contracts: Gemini function declarations, completion statuses, function calling, context caching, token counting, Gemini 3.8 Flash.