Compare commits

...

21 Commits

Author SHA1 Message Date
syntaxbullet
38565382c3 feat: add feather brush tool with adjustable settings and blending functionality
- Implemented feather brush tool in the brushRaster module, allowing for feathered edges in brush strokes.
- Added new FeatherControls component for UI adjustments of feather settings including size, radius, strength, and smoothing.
- Updated brush preview logic to accommodate feather tool alongside existing brush and eraser tools.
- Enhanced layer rendering to support feather mask previews and interactions.
- Introduced blending logic for feathered strokes to mix blurred mask values with original pixels.
- Added unit tests for feather blending functionality and tool keybindings.
- Updated cursor handling to reflect feather tool usage.
2026-07-11 22:08:25 +02:00
syntaxbullet
95043dfbdd feat: enhance ControlNet integration with Xinsir-compatible pose rendering and update model handling 2026-07-11 19:46:02 +02:00
syntaxbullet
ff762b8f17 feat: add inpaint region functionality and related tools
- Enhanced cursor behavior for new tools: semantic select, mask lasso, and mask rectangle.
- Updated mask edit state to include mask asset ID and kind.
- Implemented inpaint region commands for adding, applying, and removing inpaint regions.
- Introduced new operations for lasso and semantic selection tools.
- Created UI components for candidate review and inpaint region management.
- Added tests for inpaint region commands to ensure functionality.
- Updated various components to support new inpaint features and improve user experience.
2026-07-11 16:41:22 +02:00
syntaxbullet
f4e13b80e7 feat: add script to launch Image Studio with local ComfyUI server 2026-07-11 15:11:22 +02:00
syntaxbullet
5915c62a9a refactor: Update bottom controls for improved styling and functionality
- Adjusted button styles across various components for consistency and better UX.
- Enhanced layout of action controls to utilize whitespace more effectively.
- Updated slider styles for a more modern appearance and improved usability.
- Refined input fields and labels for better accessibility and readability.
- Introduced new app surface styles for a cohesive design across the application.
- Added tests for canvas cursor behavior to ensure correct cursor display during operations.
2026-07-11 14:55:32 +02:00
syntaxbullet
37aa719047 feat: add first-class text layers 2026-07-11 12:38:34 +02:00
syntaxbullet
606426c885 feat: add non-destructive adjustment layers 2026-07-11 12:31:06 +02:00
syntaxbullet
b928fb599b feat: add layer and mask thumbnails 2026-07-11 12:20:26 +02:00
syntaxbullet
9fb3a19912 feat: add crop and canvas resize workflows 2026-07-11 12:16:33 +02:00
syntaxbullet
4d7358af4b feat: add contextual layer inspector 2026-07-11 12:07:36 +02:00
syntaxbullet
bef2fb3051 feat: unify AI edit workflow 2026-07-11 11:58:16 +02:00
syntaxbullet
49b029d60a feat: update project name to 'image-studio' and enhance document title in App component 2026-07-11 11:38:59 +02:00
syntaxbullet
03493a1c32 feat: add generation cancel job functionality and improve job handling
- Introduced `generationCancelJob` command ID in `commands/ids.ts`.
- Added `GenerationCancelJobPayload` type in `commands/payloads.ts`.
- Enhanced job status to include "cancelled" in `editor/state.ts`.
- Updated `runGenerationJob` to accept an `AbortSignal` and handle cancellation.
- Implemented cancellation logic in `runGenerate` and related functions.
- Added tests for job cancellation in `operations/generation/workflow.test.ts`.
- Improved layer rendering logic to prevent stack overflow in `editor/document-indexes.ts`.
- Added raster size assertions in `platform/browser/rasterLimits.ts` for image processing limits.
- Enhanced image file handling to check for size limits in `platform/browser/imageFiles.ts`.
- Updated UI components to reflect job cancellation state in `view/GenerationJobStatus.tsx` and `view/bottom-controls/GenerateActionControls.tsx`.
2026-07-11 11:37:21 +02:00
syntaxbullet
d8fdd43416 feat: enhance accessibility by adding ARIA attributes and focus styles; improve canvas and button elements 2026-07-11 11:28:57 +02:00
syntaxbullet
53cf25c132 feat: implement document actions for managing artboards, zoom, and history; update UI components to utilize new actions 2026-07-11 11:26:27 +02:00
syntaxbullet
1236b6dd2f feat: add commands for clearing candidates and reusing candidate settings, update UI controls 2026-07-11 11:23:25 +02:00
syntaxbullet
556fd685a2 feat: implement chroma key operation and related keybinds, update workspace panel management 2026-07-11 00:09:41 +02:00
syntaxbullet
51a54dbdb2 feat: implement generation workflow with resource loading and candidate management 2026-07-10 23:54:52 +02:00
syntaxbullet
9697075d29 feat: add project management features including open/save functionality and recovery support 2026-07-10 23:46:01 +02:00
syntaxbullet
6d652c3264 feat: implement resolveGeneratedOutputPlacement function and add tests for output placement logic 2026-07-10 23:29:03 +02:00
syntaxbullet
06f2898080 feat: implement generation preconditions check and integrate with generation commands 2026-07-10 23:25:24 +02:00
149 changed files with 6227 additions and 965 deletions

View File

@@ -5,6 +5,8 @@ Image Studio is a Bun + React image editor prototype. It uses a command-driven a
## Features
- Import images into an artboard
- Save and open versioned project files with embedded, project-owned assets
- Automatic local recovery snapshots after document changes
- Export the active artboard as PNG
- Layer selection and layer sheet
- Select, transform, pan, brush, eraser, chroma key, and magic wand tools
@@ -34,6 +36,19 @@ Start the development server:
bun dev
```
To start Image Studio together with the local headless ComfyUI server:
```bash
bun run dev:full
```
The combined launcher reuses a ComfyUI server already listening at
`http://127.0.0.1:8188`. Otherwise, it starts the existing local ComfyUI
installation and stops that process when Image Studio exits. Its paths and
address can be overridden with the `COMFYUI_BASE_DIR`, `COMFYUI_SOURCE_DIR`,
`COMFYUI_PYTHON`, `COMFYUI_MODEL_PATHS_CONFIG`, `COMFYUI_HOST`, and
`COMFYUI_PORT` environment variables.
Build for production:
```bash
@@ -51,6 +66,7 @@ bun start
| Command | Description |
| --- | --- |
| `bun dev` | Start the app with Bun hot reload |
| `bun run dev:full` | Start the app and local headless ComfyUI server |
| `bun run build` | Build the production bundle |
| `bun start` | Run the production server |
| `bun test` | Run tests |
@@ -71,6 +87,8 @@ bun start
| `Space` | Hold to pan |
| `L` | Toggle layers |
| `Cmd/Ctrl` + `O` | Open image |
| `Shift` + `Cmd/Ctrl` + `O` | Open project |
| `Cmd/Ctrl` + `S` | Save project |
| `Cmd/Ctrl` + `Z` | Undo |
| `Shift` + `Cmd/Ctrl` + `Z` | Redo |
| `Delete` / `Backspace` | Delete selection |

View File

@@ -10,14 +10,20 @@ import { transformCommands } from "@commands/transform";
import { viewportCommands } from "@commands/viewport";
import { workspaceCommands } from "@commands/workspace";
import { editorCommands } from "@commands/editor";
import { projectCommands } from "@commands/project";
import { inpaintRegionCommands } from "@commands/inpaint-region";
import { createInitialAppState } from "@editor/initial-state";
import { createAppStore } from "@editor/store";
import { createGenerationWorkflow } from "@operations/generation/workflow";
import { createDocumentActions } from "./document-actions";
export type ImageStudioApp = ReturnType<typeof createImageStudioApp>;
export function createImageStudioApp(options?: { documentName?: string; createDefaultArtboard?: boolean }) {
const registry = createCommandRegistry([...viewportCommands, ...selectionCommands, ...documentCommands, ...toolCommands, ...generationCommands, ...transformCommands, ...historyCommands, ...commandPaletteCommands, ...workspaceCommands, ...editorCommands]);
const registry = createCommandRegistry([...projectCommands, ...viewportCommands, ...selectionCommands, ...documentCommands, ...inpaintRegionCommands, ...toolCommands, ...generationCommands, ...transformCommands, ...historyCommands, ...commandPaletteCommands, ...workspaceCommands, ...editorCommands]);
const store = createAppStore(createInitialAppState(options?.documentName), registry);
const generation = createGenerationWorkflow(store);
const documentActions = createDocumentActions(store, generation);
if (options?.createDefaultArtboard !== false) {
const artboardId = crypto.randomUUID();
@@ -26,11 +32,13 @@ export function createImageStudioApp(options?: { documentName?: string; createDe
name: "Artboard 1",
bounds: { x: -400, y: -300, w: 800, h: 600 },
});
store.dispatch(commandIds.viewportFitArtboard, { artboardId });
documentActions.fitArtboard(artboardId);
}
return {
registry,
store,
actions: { document: documentActions },
workflows: { generation },
};
}

View File

@@ -0,0 +1,34 @@
import { describe, expect, test } from "bun:test";
import { commandIds } from "@commands/ids";
import { createImageStudioApp } from "./app";
describe("document actions", () => {
test("share zoom, fit, and history behavior across UI entry points", () => {
const app = createImageStudioApp();
const artboard = app.store.getState().document.artboards[0];
if (!artboard) throw new Error("Expected the default artboard.");
app.actions.document.zoomTo(2);
app.actions.document.zoomBy(0.5);
expect(app.store.getState().editor.viewport.zoom).toBe(1);
app.actions.document.fitArtboard(artboard.id);
expect(app.store.getState().editor.viewport.center).toEqual({ x: 0, y: 0 });
expect(app.actions.document.canUndo()).toBe(true);
app.actions.document.undo();
expect(app.store.getState().document.artboards).toHaveLength(0);
expect(app.actions.document.canRedo()).toBe(true);
app.actions.document.redo();
expect(app.store.getState().document.artboards).toHaveLength(1);
});
test("opens Generate through the canonical workspace command", () => {
const app = createImageStudioApp({ createDefaultArtboard: false });
app.actions.document.openGenerate();
expect(app.store.getState().editor.workspace.panel).toBe("generate");
app.store.dispatch(commandIds.workspaceSetPanel, { panel: "none" });
expect(app.store.getState().editor.workspace.panel).toBe("none");
});
});

58
app/document-actions.ts Normal file
View File

@@ -0,0 +1,58 @@
import { commandIds } from "@commands/ids";
import type { ArtboardId } from "@core/id";
import type { AppStore } from "@editor/store";
import { downloadArtboardPng } from "@operations/export/downloadArtboard";
import type { GenerationWorkflow } from "@operations/generation/workflow";
export type DocumentActions = ReturnType<typeof createDocumentActions>;
export function createDocumentActions(store: AppStore, generation: GenerationWorkflow) {
return {
openGenerate() {
store.dispatch(commandIds.workspaceSetPanel, { panel: "generate" });
},
generate() {
return generation.generate();
},
exportArtboard(artboardId?: ArtboardId) {
const state = store.getState();
const resolvedId = artboardId ?? state.editor.selection.artboardId;
const artboard = resolvedId
? state.document.artboards.find((candidate) => candidate.id === resolvedId)
: state.document.artboards[0];
if (!artboard) return Promise.resolve();
return downloadArtboardPng(artboard, state.document.assets);
},
fitArtboard(artboardId?: ArtboardId) {
store.dispatch(commandIds.viewportFitArtboard, artboardId ? { artboardId } : undefined);
},
zoomBy(factor: number) {
const zoom = store.getState().editor.viewport.zoom;
store.dispatch(commandIds.viewportSetZoom, { zoom: zoom * factor });
},
zoomTo(zoom: number) {
store.dispatch(commandIds.viewportSetZoom, { zoom });
},
undo() {
store.dispatch(commandIds.historyUndo, undefined);
},
canUndo() {
return store.getState().history.past.length > 0;
},
redo() {
store.dispatch(commandIds.historyRedo, undefined);
},
canRedo() {
return store.getState().history.future.length > 0;
},
};
}

View File

@@ -1,2 +1,3 @@
export type { ImageStudioApp } from "./app";
export { createImageStudioApp } from "./app";
export type { DocumentActions } from "./document-actions";

View File

@@ -3,7 +3,7 @@
"configVersion": 1,
"workspaces": {
"": {
"name": "bun-react-template",
"name": "image-studio",
"dependencies": {
"@phosphor-icons/react": "^2.1.10",
"@radix-ui/react-slot": "^1.2.3",

View File

@@ -0,0 +1,39 @@
import { describe, expect, test } from "bun:test";
import { createInitialAppState } from "@editor/initial-state";
import { neutralColorAdjustment } from "@core/adjustment-layer";
import { documentAddArtboardCommand, documentAddAdjustmentLayerCommand, documentSetAdjustmentCommand, documentDuplicateLayerCommand, documentMoveLayerCommand } from "./document";
const transform = { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 };
function stateWithAdjustment() {
let state = documentAddArtboardCommand.execute({ state: createInitialAppState("test") }, { id: "board", name: "Board", bounds: { x: 0, y: 0, w: 10, h: 10 } });
state = documentAddAdjustmentLayerCommand.execute({ state }, { artboardId: "board", layer: { id: "adjust", type: "adjustment", name: "Color", visible: true, locked: false, opacity: 1, transform, adjustment: neutralColorAdjustment } });
return state;
}
describe("adjustment commands", () => {
test("creates and selects a persistent adjustment node", () => {
const state = stateWithAdjustment();
expect(state.document.artboards[0]?.layers[0]?.type).toBe("adjustment");
expect(state.editor.selection.layerIds).toEqual(["adjust"]);
});
test("validates edits and preserves locked layers", () => {
const state = stateWithAdjustment();
const changed = documentSetAdjustmentCommand.execute({ state }, { layerId: "adjust", adjustment: { ...neutralColorAdjustment, brightness: .25 } });
const invalid = documentSetAdjustmentCommand.execute({ state: changed }, { layerId: "adjust", adjustment: { ...neutralColorAdjustment, contrast: 2 } });
expect((changed.document.artboards[0]?.layers[0] as { adjustment: { brightness: number } }).adjustment.brightness).toBe(.25);
expect(invalid).toBe(changed);
});
test("duplicates and moves adjustment nodes like tree siblings", () => {
let state = stateWithAdjustment();
state = documentDuplicateLayerCommand.execute({ state }, { layerId: "adjust", idByLayerId: { adjust: "copy" } });
state = documentMoveLayerCommand.execute({ state }, { layerId: "copy", toArtboardId: "board", toIndex: 0 });
expect(state.document.artboards[0]?.layers.map((layer) => [layer.id, layer.type])).toEqual([["copy", "adjustment"], ["adjust", "adjustment"]]);
});
test("rejects nested creation, movement, and grouping to preserve exact scope", () => {
const state = stateWithAdjustment();
const nested = documentAddAdjustmentLayerCommand.execute({ state }, { artboardId: "board", parentGroupId: "group", layer: { id: "nested", type: "adjustment", name: "Nested", visible: true, locked: false, opacity: 1, transform, adjustment: neutralColorAdjustment } });
const moved = documentMoveLayerCommand.execute({ state }, { layerId: "adjust", toArtboardId: "board", toParentGroupId: "group", toIndex: 0 });
expect(nested).toBe(state);
expect(moved).toBe(state);
});
});

View File

@@ -3,6 +3,7 @@ import type { ArtboardId, LayerId } from "@core/id";
import type { Layer } from "@core/layer";
import type { LayerGroup } from "@core/layer-group";
import { getLayerMask } from "@core/layer-mask-utils";
import type { MaskEditState } from "@editor/state";
export type LayerLocation = {
artboardId: ArtboardId;
@@ -242,13 +243,18 @@ export function removeMissingMaskReferences(document: ImageDocument): ImageDocum
});
}
export function isMaskEditFor(maskEdit: { targetLayerId: LayerId; maskLayerId: LayerId } | undefined, targetLayerId: LayerId, maskLayerId: LayerId) {
export function isMaskEditFor(maskEdit: MaskEditState | undefined, targetLayerId: LayerId, maskLayerId: LayerId) {
return maskEdit?.targetLayerId === targetLayerId && maskEdit.maskLayerId === maskLayerId;
}
export function isMaskEditValid(maskEdit: { targetLayerId: LayerId; maskLayerId: LayerId } | undefined, document: ImageDocument) {
export function isMaskEditValid(maskEdit: MaskEditState | undefined, document: ImageDocument) {
if (!maskEdit) return false;
const target = findLayerLocation(document, maskEdit.targetLayerId)?.layer;
if (maskEdit.kind === "inpaintRegion") {
const region = document.inpaintRegions.find((candidate) => candidate.id === maskEdit.inpaintRegionId && candidate.targetLayerId === maskEdit.targetLayerId && candidate.maskAssetId === maskEdit.maskAssetId);
return Boolean(target && region && document.assets.some((asset) => asset.id === region.maskAssetId));
}
if (!maskEdit.maskLayerId) return false;
const mask = findLayerLocation(document, maskEdit.maskLayerId)?.layer;
return Boolean(target && getLayerMask(target)?.maskLayerId === maskEdit.maskLayerId && mask && mask.type !== "group");
}
@@ -300,3 +306,26 @@ export function mapAllLayersInTree(layers: Layer[], mapLayer: (layer: Layer) =>
return mapLayer(mapped);
});
}
export function removeInpaintRegionsForTargets(document: ImageDocument, targetLayerIds: ReadonlySet<LayerId>): ImageDocument {
const removed = document.inpaintRegions.filter((region) => targetLayerIds.has(region.targetLayerId));
if (removed.length === 0) return document;
const inpaintRegions = document.inpaintRegions.filter((region) => !targetLayerIds.has(region.targetLayerId));
const removedMaskAssetIds = new Set(removed.map((region) => region.maskAssetId));
const retainedRegionAssetIds = new Set(inpaintRegions.map((region) => region.maskAssetId));
const layerAssetIds = new Set<string>();
for (const artboard of document.artboards) {
const stack = [...artboard.layers];
while (stack.length > 0) {
const layer = stack.pop();
if (!layer) continue;
if (layer.type === "group") stack.push(...layer.children);
else if (layer.type === "image" || layer.type === "raster") layerAssetIds.add(layer.assetId);
}
}
return {
...document,
inpaintRegions,
assets: document.assets.filter((asset) => !removedMaskAssetIds.has(asset.id) || retainedRegionAssetIds.has(asset.id) || layerAssetIds.has(asset.id)),
};
}

View File

@@ -10,6 +10,7 @@ import {
documentAddRasterLayerCommand,
documentApplyLayerMaskOperationCommand,
documentGroupLayersCommand,
documentDuplicateLayerCommand,
documentMoveLayerCommand,
documentRemoveArtboardCommand,
documentRemoveLayerCommand,
@@ -17,10 +18,13 @@ import {
documentRenameArtboardCommand,
documentRenameLayerCommand,
documentSetArtboardBoundsCommand,
documentResizeArtboardCommand,
documentSetArtboardLockedCommand,
documentSetArtboardVisibleCommand,
documentSetLayerClippingMaskCommand,
documentSetLayerLockedCommand,
documentSetLayerOpacityCommand,
documentSetLayerSourceRectCommand,
documentSetLayerVisibleCommand,
documentUpdateAssetSourceCommand,
documentUngroupLayerCommand,
@@ -57,6 +61,32 @@ describe("document commands", () => {
]);
});
test("crops a raster layer non-destructively and clamps the crop to its asset", () => {
const state = documentWithRaster();
const next = documentSetLayerSourceRectCommand.execute({ state }, { layerId: "r1", sourceRect: { x: 20, y: 10, w: 200, h: 100 } });
expect(next.document.artboards[0]?.layers[0]).toMatchObject({ sourceRect: { x: 20, y: 10, w: 80, h: 40 } });
expect(next.document.assets[0]?.intrinsicSize).toEqual({ w: 100, h: 50 });
const reset = documentSetLayerSourceRectCommand.execute({ state: next }, { layerId: "r1" });
expect(reset.document.artboards[0]?.layers[0]).not.toHaveProperty("sourceRect");
});
test("resizes artboard bounds without scaling contents", () => {
const state = documentWithRaster();
const next = documentResizeArtboardCommand.execute({ state }, { id: "a1", bounds: { x: 5, y: 10, w: 640, h: 480 }, scaleContents: false });
expect(next.document.artboards[0]?.bounds).toEqual({ x: 5, y: 10, w: 640, h: 480 });
expect(next.document.artboards[0]?.layers[0]?.transform).toEqual(state.document.artboards[0]?.layers[0]?.transform);
});
test("resizes an artboard and scales nested contents including masks", () => {
const state = documentWithRaster();
const leaf = state.document.artboards[0]!.layers[0]!;
state.document.artboards[0]!.layers = [{ ...group("g", "Group"), children: [leaf] }];
const next = documentResizeArtboardCommand.execute({ state }, { id: "a1", bounds: { x: 10, y: 20, w: 640, h: 120 }, scaleContents: true });
const nested = next.document.artboards[0]?.layers[0];
expect(nested?.type === "group" ? nested.children[0]?.transform : undefined).toEqual({ position: { x: 30, y: 30 }, scale: { x: 2, y: 0.5 }, rotation: 0 });
});
test("updates asset sources", () => {
const state = documentAddAssetCommand.execute(
{ state: createInitialAppState("Test") },
@@ -255,7 +285,7 @@ describe("document commands", () => {
inverted: false,
});
expect(masked.document.artboards[0]?.layers[1]?.clippingMask).toEqual({ maskLayerId: "mask" });
expect(masked.editor.maskEdit).toEqual({ targetLayerId: "target", maskLayerId: "mask" });
expect(masked.editor.maskEdit).toEqual({ kind: "layerMask", targetLayerId: "target", maskLayerId: "mask", maskAssetId: "mask-asset" });
expect(masked.editor.tools.activeTool).toBe("brush");
});
@@ -335,6 +365,46 @@ describe("document commands", () => {
expect(locked.document.artboards[0]?.layers[0]?.locked).toBe(true);
});
test("sets normalized opacity and refuses locked layer edits", () => {
const state = documentWithLayers([raster("a", "A")]);
const translucent = documentSetLayerOpacityCommand.execute({ state }, { layerId: "a", opacity: 0.35 });
const clamped = documentSetLayerOpacityCommand.execute({ state: translucent }, { layerId: "a", opacity: 2 });
const lockedState = documentSetLayerLockedCommand.execute({ state: clamped }, { layerId: "a", locked: true });
const ignored = documentSetLayerOpacityCommand.execute({ state: lockedState }, { layerId: "a", opacity: 0 });
expect(translucent.document.artboards[0]?.layers[0]?.opacity).toBe(0.35);
expect(clamped.document.artboards[0]?.layers[0]?.opacity).toBe(1);
expect(ignored).toBe(lockedState);
});
test("duplicates a nested group with remapped child mask references", () => {
const maskedTarget = { ...raster("target", "Target"), layerMask: { kind: "raster" as const, maskLayerId: "mask", enabled: true, inverted: false } };
const parent = { ...group("parent", "Parent"), children: [raster("mask", "Mask"), maskedTarget] };
const state = documentWithLayers([parent]);
const next = documentDuplicateLayerCommand.execute(
{ state },
{ layerId: "parent", idByLayerId: { parent: "parent-copy", mask: "mask-copy", target: "target-copy" } },
);
const duplicate = next.document.artboards[0]?.layers[1];
expect(next.document.artboards[0]?.layers.map((layer) => layer.id)).toEqual(["parent", "parent-copy"]);
expect(duplicate?.type === "group" ? duplicate.children.map((layer) => layer.id) : []).toEqual(["mask-copy", "target-copy"]);
expect(duplicate?.type === "group" ? duplicate.children[1]?.layerMask?.maskLayerId : undefined).toBe("mask-copy");
expect(next.editor.selection.layerIds).toEqual(["parent-copy"]);
});
test("duplicates an attached top-level mask beside its target", () => {
const target = { ...raster("target", "Target"), layerMask: { kind: "raster" as const, maskLayerId: "mask", enabled: true, inverted: false } };
const state = documentWithLayers([raster("mask", "Mask"), target]);
const next = documentDuplicateLayerCommand.execute(
{ state },
{ layerId: "target", idByLayerId: { mask: "mask-copy", target: "target-copy" } },
);
expect(next.document.artboards[0]?.layers.map((layer) => layer.id)).toEqual(["mask", "target", "mask-copy", "target-copy"]);
expect(next.document.artboards[0]?.layers[3]?.layerMask?.maskLayerId).toBe("mask-copy");
});
test("sets artboard bounds", () => {
const state = documentAddArtboardCommand.execute(
{ state: createInitialAppState("Test") },
@@ -361,6 +431,12 @@ function documentWithLayers(layers: Layer[]) {
};
}
function documentWithRaster() {
const state = documentWithLayers([{ ...raster("r1", "Raster", "asset-1"), transform: { position: { x: 10, y: 20 }, scale: { x: 1, y: 1 }, rotation: 0 } }]);
state.document.assets = [{ id: "asset-1", name: "Raster", mimeType: "image/png", source: "asset://raster", intrinsicSize: { w: 100, h: 50 } }];
return state;
}
function group(id: string, name: string) {
return {
id,

View File

@@ -1,4 +1,4 @@
import { findLayerLocation, isReferencedMaskLayer, mapLayerInDocument, insertLayer, replaceLayerListInDocument, replaceSelectedLayersWithGroup, removeLayerFromDocument, ungroupLayerInDocument, findGroup, removeLayerMaskReference, withLayerMask, removeUnreferencedMaskLayer, removeMissingMaskReferences, isMaskEditFor, isMaskEditValid, collectLayerIds, collectClippingMaskIds, collectAttachedMaskIds } from "./document-tree";
import { findLayerLocation, isReferencedMaskLayer, mapLayerInDocument, insertLayer, replaceLayerListInDocument, replaceSelectedLayersWithGroup, removeLayerFromDocument, ungroupLayerInDocument, findGroup, removeLayerMaskReference, withLayerMask, removeUnreferencedMaskLayer, removeMissingMaskReferences, isMaskEditFor, isMaskEditValid, collectLayerIds, collectClippingMaskIds, collectAttachedMaskIds, removeInpaintRegionsForTargets } from "./document-tree";
import type { Asset } from "@core/asset";
import type { ImageDocument } from "@core/document";
import type { Rect } from "@core/geometry";
@@ -6,7 +6,11 @@ import type { ArtboardId, AssetId, LayerId } from "@core/id";
import type { ImageLayer } from "@core/image-layer";
import { getLayerMask } from "@core/layer-mask-utils";
import type { RasterLayer } from "@core/raster-layer";
import type { Layer } from "@core/layer";
import type { LayerGroup } from "@core/layer-group";
import type { AdjustmentLayer, ColorAdjustment } from "@core/adjustment-layer";
import type { TextLayer, TextStyle } from "@core/text-layer";
import { isValidTextStyle } from "@core/text-layer";
import type { Command } from "./command";
import { commandIds } from "./ids";
@@ -21,6 +25,12 @@ export type DocumentSetArtboardBoundsPayload = {
bounds: Rect;
};
export type DocumentResizeArtboardPayload = {
id: ArtboardId;
bounds: Rect;
scaleContents: boolean;
};
export type DocumentRemoveArtboardPayload = {
id: ArtboardId;
};
@@ -66,6 +76,10 @@ export type DocumentAddGroupLayerPayload = {
parentGroupId?: LayerId;
group: LayerGroup;
};
export type DocumentAddAdjustmentLayerPayload = { artboardId: ArtboardId; parentGroupId?: LayerId; layer: AdjustmentLayer };
export type DocumentSetAdjustmentPayload = { layerId: LayerId; adjustment: ColorAdjustment };
export type DocumentAddTextLayerPayload = { artboardId: ArtboardId; parentGroupId?: LayerId; layer: TextLayer };
export type DocumentSetTextLayerPayload = { layerId: LayerId; content: string; style: TextStyle };
export type DocumentMoveLayerPayload = {
layerId: LayerId;
@@ -98,6 +112,21 @@ export type DocumentSetLayerLockedPayload = {
locked: boolean;
};
export type DocumentSetLayerOpacityPayload = {
layerId: LayerId;
opacity: number;
};
export type DocumentSetLayerSourceRectPayload = {
layerId: LayerId;
sourceRect?: Rect;
};
export type DocumentDuplicateLayerPayload = {
layerId: LayerId;
idByLayerId: Record<LayerId, LayerId>;
};
export type DocumentRenameLayerPayload = {
layerId: LayerId;
name: string;
@@ -112,6 +141,7 @@ export type DocumentAddLayerMaskPayload = {
layerId: LayerId;
asset: Asset;
maskLayer: RasterLayer;
activeTool?: "brush" | "feather";
};
export type LayerMaskOperation =
@@ -178,15 +208,42 @@ export const documentSetArtboardBoundsCommand: Command<DocumentSetArtboardBounds
},
};
export const documentResizeArtboardCommand: Command<DocumentResizeArtboardPayload> = {
id: commandIds.documentResizeArtboard,
name: "Resize artboard",
execute({ state }, payload) {
const artboard = state.document.artboards.find((candidate) => candidate.id === payload.id);
const bounds = validRect(payload.bounds);
if (!artboard || artboard.locked || !bounds) return state;
const scaleX = bounds.w / artboard.bounds.w;
const scaleY = bounds.h / artboard.bounds.h;
return {
...state,
document: {
...state.document,
artboards: state.document.artboards.map((candidate) => candidate.id !== payload.id ? candidate : {
...candidate,
bounds,
layers: payload.scaleContents ? scaleLayerTree(candidate.layers, artboard.bounds, bounds, scaleX, scaleY) : candidate.layers,
}),
},
};
},
};
export const documentRemoveArtboardCommand: Command<DocumentRemoveArtboardPayload> = {
id: commandIds.documentRemoveArtboard,
name: "Remove artboard",
execute({ state }, payload) {
const removedSelectedArtboard = state.editor.selection.artboardId === payload.id;
const document = {
const removedArtboard = state.document.artboards.find((artboard) => artboard.id === payload.id);
const withoutArtboard = {
...state.document,
artboards: state.document.artboards.filter((artboard) => artboard.id !== payload.id),
};
const removedLayerIds = new Set<LayerId>();
for (const layer of removedArtboard?.layers ?? []) collectLayerIds(layer, removedLayerIds);
const document = removeInpaintRegionsForTargets(withoutArtboard, removedLayerIds);
return {
...state,
@@ -308,10 +365,51 @@ export const documentAddGroupLayerCommand: Command<DocumentAddGroupLayerPayload>
},
};
export const documentAddAdjustmentLayerCommand: Command<DocumentAddAdjustmentLayerPayload> = {
id: commandIds.documentAddAdjustmentLayer,
name: "Add adjustment layer",
execute({ state }, payload) {
if (payload.parentGroupId || !validAdjustment(payload.layer.adjustment)) return state;
return { ...state, document: insertLayer(state.document, payload.artboardId, undefined, payload.layer, 0), editor: { ...state.editor, selection: { artboardId: payload.artboardId, layerIds: [payload.layer.id] } } };
},
};
export const documentAddTextLayerCommand: Command<DocumentAddTextLayerPayload> = {
id: commandIds.documentAddTextLayer,
name: "Add text layer",
execute({ state }, payload) {
if (!payload.layer.content.trim() || !isValidTextStyle(payload.layer.style)) return state;
const document = insertLayer(state.document, payload.artboardId, payload.parentGroupId, payload.layer);
return { ...state, document, editor: { ...state.editor, selection: { artboardId: payload.artboardId, layerIds: [payload.layer.id] } } };
},
};
export const documentSetTextLayerCommand: Command<DocumentSetTextLayerPayload> = {
id: commandIds.documentSetTextLayer,
name: "Edit text layer",
execute({ state }, payload) {
const location = findLayerLocation(state.document, payload.layerId);
if (!location || location.layer.type !== "text" || location.layer.locked || !payload.content.trim() || !isValidTextStyle(payload.style)) return state;
return { ...state, document: mapLayerInDocument(state.document, payload.layerId, (layer) => layer.type === "text" ? { ...layer, content: payload.content, style: { ...payload.style } } : layer) };
},
};
export const documentSetAdjustmentCommand: Command<DocumentSetAdjustmentPayload> = {
id: commandIds.documentSetAdjustment,
name: "Edit adjustment layer",
execute({ state }, payload) {
const location = findLayerLocation(state.document, payload.layerId);
if (!location || location.layer.type !== "adjustment" || location.layer.locked || !validAdjustment(payload.adjustment)) return state;
return { ...state, document: mapLayerInDocument(state.document, payload.layerId, (layer) => layer.type === "adjustment" ? { ...layer, adjustment: payload.adjustment } : layer) };
},
};
export const documentMoveLayerCommand: Command<DocumentMoveLayerPayload> = {
id: commandIds.documentMoveLayer,
name: "Move layer",
execute({ state }, payload) {
const source = findLayerLocation(state.document, payload.layerId);
if (source?.layer.type === "adjustment" && payload.toParentGroupId) return state;
if (payload.toParentGroupId && !findGroup(state.document, payload.toParentGroupId)) return state;
const removed = removeLayerFromDocument(state.document, payload.layerId);
@@ -345,6 +443,7 @@ export const documentGroupLayersCommand: Command<DocumentGroupLayersPayload> = {
const location = findLayerLocation(state.document, layerId);
return location ? [location] : [];
});
if (requestedLocations.some((location) => location.layer.type === "adjustment")) return state;
if (requestedLocations.length !== requestedIds.length) return state;
const firstLocation = requestedLocations[0];
@@ -402,6 +501,77 @@ export const documentSetLayerLockedCommand: Command<DocumentSetLayerLockedPayloa
},
};
export const documentSetLayerOpacityCommand: Command<DocumentSetLayerOpacityPayload> = {
id: commandIds.documentSetLayerOpacity,
name: "Set layer opacity",
execute({ state }, payload) {
const location = findLayerLocation(state.document, payload.layerId);
if (!location || location.layer.locked || !Number.isFinite(payload.opacity)) return state;
const opacity = Math.min(1, Math.max(0, payload.opacity));
return { ...state, document: mapLayerInDocument(state.document, payload.layerId, (layer) => ({ ...layer, opacity })) };
},
};
export const documentSetLayerSourceRectCommand: Command<DocumentSetLayerSourceRectPayload> = {
id: commandIds.documentSetLayerSourceRect,
name: "Crop layer",
execute({ state }, payload) {
const location = findLayerLocation(state.document, payload.layerId);
if (!location || (location.layer.type !== "image" && location.layer.type !== "raster") || location.layer.locked || location.layer.transform.rotation !== 0) return state;
const leaf = location.layer;
const asset = state.document.assets.find((candidate) => candidate.id === leaf.assetId);
if (!asset) return state;
const sourceRect = payload.sourceRect ? clampSourceRect(payload.sourceRect, asset.intrinsicSize.w, asset.intrinsicSize.h) : undefined;
if (payload.sourceRect && !sourceRect) return state;
return { ...state, document: mapLayerInDocument(state.document, payload.layerId, (layer) => {
if (layer.type !== "image" && layer.type !== "raster") return layer;
if (sourceRect) return { ...layer, sourceRect };
const uncropped = { ...layer };
delete uncropped.sourceRect;
return uncropped;
}) };
},
};
export const documentDuplicateLayerCommand: Command<DocumentDuplicateLayerPayload> = {
id: commandIds.documentDuplicateLayer,
name: "Duplicate layer",
execute({ state }, payload) {
const location = findLayerLocation(state.document, payload.layerId);
if (!location || location.layer.locked) return state;
const maskId = getLayerMask(location.layer)?.maskLayerId;
const mask = maskId ? location.siblings.find((layer) => layer.id === maskId) : undefined;
const sourceLayers = mask ? [mask, location.layer] : [location.layer];
const sourceIds = new Set(sourceLayers.flatMap((layer) => [...collectLayerIds(layer)]));
const mappedIds = [...sourceIds].map((id) => payload.idByLayerId[id]);
if (mappedIds.some((id) => !id) || new Set(mappedIds).size !== mappedIds.length || mappedIds.some((id) => findLayerLocation(state.document, id!))) return state;
const duplicates = sourceLayers.map((layer) => duplicateLayerTree(layer, payload.idByLayerId));
const insertionIndex = Math.max(...sourceLayers.map((layer) => location.siblings.findIndex((candidate) => candidate.id === layer.id))) + 1;
const siblings = [...location.siblings.slice(0, insertionIndex), ...duplicates, ...location.siblings.slice(insertionIndex)];
const duplicatedLayerId = payload.idByLayerId[payload.layerId];
if (!duplicatedLayerId) return state;
return {
...state,
document: replaceLayerListInDocument(state.document, location.artboardId, location.parentGroupId, siblings),
editor: { ...state.editor, selection: { artboardId: location.artboardId, layerIds: [duplicatedLayerId] } },
};
},
};
function duplicateLayerTree(layer: Layer, idByLayerId: Record<LayerId, LayerId>): Layer {
const layerMask = getLayerMask(layer);
const duplicatedMaskId = layerMask ? idByLayerId[layerMask.maskLayerId] : undefined;
const duplicated = {
...layer,
id: idByLayerId[layer.id]!,
name: `${layer.name} copy`,
transform: { ...layer.transform, position: { ...layer.transform.position }, scale: { ...layer.transform.scale } },
...(layerMask && duplicatedMaskId ? { layerMask: { ...layerMask, maskLayerId: duplicatedMaskId }, clippingMask: layer.clippingMask ? { maskLayerId: duplicatedMaskId } : undefined } : {}),
};
return layer.type === "group" ? { ...duplicated, type: "group", children: layer.children.map((child) => duplicateLayerTree(child, idByLayerId)) } : duplicated;
}
export const documentRenameLayerCommand: Command<DocumentRenameLayerPayload> = {
id: commandIds.documentRenameLayer,
name: "Rename layer",
@@ -433,6 +603,7 @@ export const documentSetLayerClippingMaskCommand: Command<DocumentSetLayerClippi
const targetLocation = findLayerLocation(state.document, payload.layerId);
const maskLocation = findLayerLocation(state.document, payload.maskLayerId);
if (!targetLocation || !maskLocation) return state;
if (targetLocation.layer.type === "adjustment" || targetLocation.layer.type === "text") return state;
if (targetLocation.artboardId !== maskLocation.artboardId || targetLocation.parentGroupId !== maskLocation.parentGroupId) return state;
const removed = removeLayerFromDocument(state.document, payload.layerId);
@@ -459,23 +630,24 @@ export const documentAddLayerMaskCommand: Command<DocumentAddLayerMaskPayload> =
name: "Add layer mask",
execute({ state }, payload) {
const targetLocation = findLayerLocation(state.document, payload.layerId);
if (!targetLocation || targetLocation.layer.type === "group") return state;
if (!targetLocation || (targetLocation.layer.type !== "image" && targetLocation.layer.type !== "raster")) return state;
const existingMaskId = getLayerMask(targetLocation.layer)?.maskLayerId;
if (existingMaskId) {
const existingMaskLocation = findLayerLocation(state.document, existingMaskId);
if (existingMaskLocation?.layer.type === "group") return state;
if (existingMaskLocation) {
const existingMaskLayer = existingMaskLocation?.layer;
if (existingMaskLayer && existingMaskLayer.type !== "image" && existingMaskLayer.type !== "raster") return state;
if (existingMaskLocation && existingMaskLayer) {
return {
...state,
editor: {
...state.editor,
selection: { artboardId: targetLocation.artboardId, layerIds: [payload.layerId] },
maskEdit: { targetLayerId: payload.layerId, maskLayerId: existingMaskId },
maskEdit: { kind: "layerMask", targetLayerId: payload.layerId, maskLayerId: existingMaskId, maskAssetId: existingMaskLayer.assetId },
tools: {
...state.editor.tools,
activeTool: "brush",
interactionMode: { type: "tool", tool: "brush" },
activeTool: payload.activeTool ?? "brush",
interactionMode: { type: "tool", tool: payload.activeTool ?? "brush" },
},
},
};
@@ -504,11 +676,11 @@ export const documentAddLayerMaskCommand: Command<DocumentAddLayerMaskPayload> =
editor: {
...state.editor,
selection: { artboardId: targetLocation.artboardId, layerIds: [payload.layerId] },
maskEdit: { targetLayerId: payload.layerId, maskLayerId: maskLayer.id },
maskEdit: { kind: "layerMask", targetLayerId: payload.layerId, maskLayerId: maskLayer.id, maskAssetId: payload.asset.id },
tools: {
...state.editor.tools,
activeTool: "brush",
interactionMode: { type: "tool", tool: "brush" },
activeTool: payload.activeTool ?? "brush",
interactionMode: { type: "tool", tool: payload.activeTool ?? "brush" },
},
},
};
@@ -522,7 +694,7 @@ export const documentApplyLayerMaskOperationCommand: Command<DocumentApplyLayerM
if (!payload.source.trim()) return state;
const maskLocation = findLayerLocation(state.document, payload.maskLayerId);
if (!maskLocation || maskLocation.layer.type === "group") return state;
if (!maskLocation || (maskLocation.layer.type !== "image" && maskLocation.layer.type !== "raster")) return state;
if (!isReferencedMaskLayer(state.document, payload.maskLayerId)) return state;
const maskAssetId = maskLocation.layer.assetId;
@@ -583,7 +755,8 @@ export const documentRemoveLayerCommand: Command<DocumentRemoveLayerPayload> = {
const removedLayerIds = collectLayerIds(removed.layer);
const removedMaskLayerIds = collectClippingMaskIds([removed.layer]);
const cleanedReferences = removeMissingMaskReferences(removed.document);
const document = [...removedMaskLayerIds].reduce((nextDocument, maskLayerId) => removeUnreferencedMaskLayer(nextDocument, maskLayerId), cleanedReferences);
const withoutMasks = [...removedMaskLayerIds].reduce((nextDocument, maskLayerId) => removeUnreferencedMaskLayer(nextDocument, maskLayerId), cleanedReferences);
const document = removeInpaintRegionsForTargets(withoutMasks, removedLayerIds);
const selection = {
...state.editor.selection,
layerIds: state.editor.selection.layerIds.filter((id) => !removedLayerIds.has(id)),
@@ -604,6 +777,7 @@ export const documentRemoveLayerCommand: Command<DocumentRemoveLayerPayload> = {
export const documentCommands = [
documentAddArtboardCommand,
documentSetArtboardBoundsCommand,
documentResizeArtboardCommand,
documentRemoveArtboardCommand,
documentSetArtboardVisibleCommand,
documentSetArtboardLockedCommand,
@@ -613,15 +787,56 @@ export const documentCommands = [
documentAddImageLayerCommand,
documentAddRasterLayerCommand,
documentAddGroupLayerCommand,
documentAddAdjustmentLayerCommand,
documentAddTextLayerCommand,
documentSetTextLayerCommand,
documentSetAdjustmentCommand,
documentMoveLayerCommand,
documentGroupLayersCommand,
documentUngroupLayerCommand,
documentRemoveLayerCommand,
documentSetLayerVisibleCommand,
documentSetLayerLockedCommand,
documentSetLayerOpacityCommand,
documentSetLayerSourceRectCommand,
documentDuplicateLayerCommand,
documentRenameLayerCommand,
documentSetLayerClippingMaskCommand,
documentAddLayerMaskCommand,
documentApplyLayerMaskOperationCommand,
documentRemoveLayerMaskCommand,
] satisfies Command<unknown>[];
function validRect(rect: Rect): Rect | undefined {
return [rect.x, rect.y, rect.w, rect.h].every(Number.isFinite) && rect.w >= 1 && rect.h >= 1 ? { ...rect } : undefined;
}
function validAdjustment(value: ColorAdjustment): boolean {
return [value.brightness, value.contrast, value.saturation, value.colorBalance.red, value.colorBalance.green, value.colorBalance.blue].every((number) => Number.isFinite(number) && number >= -1 && number <= 1);
}
function clampSourceRect(rect: Rect, width: number, height: number): Rect | undefined {
if (![rect.x, rect.y, rect.w, rect.h].every(Number.isFinite)) return undefined;
const x = Math.max(0, Math.min(width - 1, rect.x));
const y = Math.max(0, Math.min(height - 1, rect.y));
const w = Math.min(width - x, rect.w);
const h = Math.min(height - y, rect.h);
return w >= 1 && h >= 1 ? { x, y, w, h } : undefined;
}
function scaleLayerTree(layers: Layer[], before: Rect, after: Rect, scaleX: number, scaleY: number): Layer[] {
return layers.map((layer) => layer.type === "adjustment" ? layer : layer.type === "group" ? {
...layer,
children: scaleLayerTree(layer.children, before, after, scaleX, scaleY),
} : {
...layer,
transform: {
...layer.transform,
position: {
x: after.x + (layer.transform.position.x - before.x) * scaleX,
y: after.y + (layer.transform.position.y - before.y) * scaleY,
},
scale: { x: layer.transform.scale.x * scaleX, y: layer.transform.scale.y * scaleY },
},
});
}

View File

@@ -5,11 +5,14 @@ import {
generationAddCandidateCommand,
generationApplyCandidateAsLayerCommand,
generationRemoveCandidateCommand,
generationClearCandidatesCommand,
generationReuseCandidateSettingsCommand,
generationReplaceCandidatePixelsCommand,
generationSetCompareModeCommand,
generationFailJobCommand,
generationStartJobCommand,
generationSucceedJobCommand,
generationCancelJobCommand,
} from "./generation";
describe("generation commands", () => {
@@ -38,6 +41,48 @@ describe("generation commands", () => {
expect(next.editor.generation.compareMode).toBe("split");
});
test("bounds retained candidate source memory as well as candidate count", () => {
const largeSource = "x".repeat(34 * 1024 * 1024);
const first = { ...generationCandidate("candidate-1"), source: largeSource };
const second = { ...generationCandidate("candidate-2"), source: largeSource };
const withFirst = generationAddCandidateCommand.execute({ state: createInitialAppState("Test") }, { candidate: first });
const withSecond = generationAddCandidateCommand.execute({ state: withFirst }, { candidate: second });
expect(withSecond.editor.generation.candidates.map((candidate) => candidate.id)).toEqual(["candidate-2"]);
});
test("clears the candidate session and resets comparison", () => {
const withCandidate = generationAddCandidateCommand.execute(
{ state: createInitialAppState("Test") },
{ candidate: generationCandidate("candidate-1") },
);
const comparing = generationSetCompareModeCommand.execute({ state: withCandidate }, { mode: "split" });
const next = generationClearCandidatesCommand.execute({ state: comparing }, undefined);
expect(next.editor.generation.candidates).toEqual([]);
expect(next.editor.generation.selectedCandidateId).toBeUndefined();
expect(next.editor.generation.compareMode).toBe("result");
});
test("reuses a candidate's full settings with its resolved seed", () => {
const candidate = generationCandidate("candidate-1");
candidate.settings = {
...candidate.settings,
prompt: "Stored prompt",
model: "stored-model.safetensors",
seed: -1,
inpaint: { ...candidate.settings.inpaint, maskBlur: 7 },
};
candidate.seed = 987654;
const state = generationAddCandidateCommand.execute({ state: createInitialAppState("Test") }, { candidate });
const next = generationReuseCandidateSettingsCommand.execute({ state }, { candidateId: candidate.id });
expect(next.editor.tools.generate).toEqual({ ...candidate.settings, seed: 987654 });
expect(next.editor.generation).toBe(state.editor.generation);
});
test("applies candidates as top-level layers", () => {
const withRemainingCandidate = generationAddCandidateCommand.execute(
{ state: documentWithSourceLayer() },
@@ -125,6 +170,13 @@ describe("generation commands", () => {
expect(failed.editor.generation.jobs[0]).toMatchObject({ id: "job-1", status: "failed", error: "Backend unavailable", finishedAt: 125 });
});
test("records explicit cancellation separately from failure", () => {
const running = generationStartJobCommand.execute({ state: createInitialAppState("Test") }, { jobId: "job-1", kind: "generate", label: "Generating", startedAt: 100 });
const cancelled = generationCancelJobCommand.execute({ state: running }, { jobId: "job-1", finishedAt: 110 });
expect(cancelled.editor.generation.jobs[0]).toMatchObject({ id: "job-1", status: "cancelled", finishedAt: 110 });
});
});
function documentWithSourceLayer() {
@@ -179,11 +231,14 @@ function generationCandidate(id: string, inpaint = false): GenerationCandidate {
maskImage: "mask",
inpaint: {
targetLayerId: "source-layer",
maskLayerId: "mask-layer",
regionId: "region",
sourceAssetId: "source-asset",
maskAssetId: "mask-asset",
inputImage: "input",
maskImage: "mask",
editMaskImage: "edit-mask",
blendMaskImage: "blend-mask",
revision: { source: "source-revision", mask: "mask-revision" },
crop: {
assetBounds: { x: 0, y: 0, w: 64, h: 64 },
documentBounds: { x: 0, y: 0, w: 64, h: 64 },

View File

@@ -23,6 +23,11 @@ export type GenerationSetCompareModePayload = {
export type GenerationRemoveCandidatePayload = {
candidateId: GenerationCandidateId;
};
export type GenerationToggleCandidateFavoritePayload = { candidateId: GenerationCandidateId };
export type GenerationReuseCandidateSettingsPayload = {
candidateId: GenerationCandidateId;
};
export type GenerationApplyCandidateAsLayerPayload = {
candidateId: GenerationCandidateId;
@@ -37,12 +42,15 @@ export type GenerationReplaceCandidatePixelsPayload = {
};
export type GenerationStartJobPayload = { jobId: GenerationJobId; kind: GenerationJobKind; label: string; startedAt: number };
export type GenerationUpdateJobPayload = { jobId: GenerationJobId; progress: number; detail: string };
export type GenerationSucceedJobPayload = { jobId: GenerationJobId; finishedAt: number };
export type GenerationFailJobPayload = { jobId: GenerationJobId; finishedAt: number; error: string };
export type GenerationCancelJobPayload = { jobId: GenerationJobId; finishedAt: number };
export type GenerationSetResourcesPayload = { options: GenerationOptions };
export type GenerationFailResourcesPayload = { error: string };
const maxCandidates = 12;
const maxCandidateSourceBytes = 96 * 1024 * 1024;
const maxJobs = 20;
const generationCompareModes = new Set<GenerationCompareMode>(["result", "before", "split"]);
@@ -51,7 +59,8 @@ export const generationAddCandidateCommand: Command<GenerationAddCandidatePayloa
name: "Add generation candidate",
history: { mode: "ignore" },
execute({ state }, payload) {
const candidates = [payload.candidate, ...state.editor.generation.candidates.filter((candidate) => candidate.id !== payload.candidate.id)].slice(0, maxCandidates);
const existing = state.editor.generation.candidates.filter((candidate) => candidate.id !== payload.candidate.id);
const candidates = retainCandidateBudget([payload.candidate, ...existing.filter((candidate) => candidate.favorite), ...existing.filter((candidate) => !candidate.favorite)]);
return {
...state,
editor: {
@@ -124,6 +133,25 @@ export const generationRemoveCandidateCommand: Command<GenerationRemoveCandidate
},
};
export const generationToggleCandidateFavoriteCommand: Command<GenerationToggleCandidateFavoritePayload> = {
id: commandIds.generationToggleCandidateFavorite,
name: "Favorite generation candidate",
history: { mode: "ignore" },
execute({ state }, payload) {
if (!state.editor.generation.candidates.some((candidate) => candidate.id === payload.candidateId)) return state;
return {
...state,
editor: {
...state.editor,
generation: {
...state.editor.generation,
candidates: state.editor.generation.candidates.map((candidate) => candidate.id === payload.candidateId ? { ...candidate, favorite: !candidate.favorite } : candidate),
},
},
};
},
};
export const generationClearCandidatesCommand: Command = {
id: commandIds.generationClearCandidates,
name: "Clear generation candidates",
@@ -140,6 +168,30 @@ export const generationClearCandidatesCommand: Command = {
},
};
export const generationReuseCandidateSettingsCommand: Command<GenerationReuseCandidateSettingsPayload> = {
id: commandIds.generationReuseCandidateSettings,
name: "Reuse generation candidate settings",
execute({ state }, payload) {
const candidate = state.editor.generation.candidates.find((item) => item.id === payload.candidateId);
if (!candidate) return state;
return {
...state,
editor: {
...state.editor,
tools: {
...state.editor.tools,
generate: {
...candidate.settings,
seed: candidate.seed,
outpaint: { ...candidate.settings.outpaint },
inpaint: { ...candidate.settings.inpaint },
},
},
},
};
},
};
export const generationApplyCandidateAsLayerCommand: Command<GenerationApplyCandidateAsLayerPayload> = {
id: commandIds.generationApplyCandidateAsLayer,
name: "Apply generation candidate as layer",
@@ -230,6 +282,17 @@ export const generationStartJobCommand: Command<GenerationStartJobPayload> = {
},
};
export const generationUpdateJobCommand: Command<GenerationUpdateJobPayload> = {
id: commandIds.generationUpdateJob,
name: "Update generation progress",
history: { mode: "ignore" },
execute({ state }, payload) {
const job = state.editor.generation.jobs.find((candidate) => candidate.id === payload.jobId);
if (!job || job.status !== "running" || !Number.isFinite(payload.progress) || !payload.detail.trim()) return state;
return updateJobs(state, state.editor.generation.jobs.map((candidate) => candidate.id === payload.jobId ? { ...candidate, progress: Math.max(0, Math.min(1, payload.progress)), detail: payload.detail.trim() } : candidate));
},
};
export const generationSucceedJobCommand: Command<GenerationSucceedJobPayload> = {
id: commandIds.generationSucceedJob,
name: "Complete generation job",
@@ -249,6 +312,15 @@ export const generationFailJobCommand: Command<GenerationFailJobPayload> = {
},
};
export const generationCancelJobCommand: Command<GenerationCancelJobPayload> = {
id: commandIds.generationCancelJob,
name: "Cancel generation job",
history: { mode: "ignore" },
execute({ state }, payload) {
return settleJob(state, payload.jobId, payload.finishedAt, "cancelled");
},
};
export const generationLoadResourcesCommand: Command = {
id: commandIds.generationLoadResources,
name: "Load generation resources",
@@ -283,12 +355,16 @@ export const generationCommands = [
generationSelectCandidateCommand,
generationSetCompareModeCommand,
generationRemoveCandidateCommand,
generationToggleCandidateFavoriteCommand,
generationClearCandidatesCommand,
generationReuseCandidateSettingsCommand,
generationApplyCandidateAsLayerCommand,
generationReplaceCandidatePixelsCommand,
generationStartJobCommand,
generationUpdateJobCommand,
generationSucceedJobCommand,
generationFailJobCommand,
generationCancelJobCommand,
generationLoadResourcesCommand,
generationSetResourcesCommand,
generationFailResourcesCommand,
@@ -302,13 +378,31 @@ function updateResources(state: AppState, resources: GenerationState["resources"
return { ...state, editor: { ...state.editor, generation: { ...state.editor.generation, resources } } };
}
function settleJob(state: AppState, jobId: GenerationJobId, finishedAt: number, status: "succeeded" | "failed", error?: string): AppState {
function settleJob(state: AppState, jobId: GenerationJobId, finishedAt: number, status: "succeeded" | "failed" | "cancelled", error?: string): AppState {
if (!Number.isFinite(finishedAt)) return state;
const job = state.editor.generation.jobs.find((candidate) => candidate.id === jobId);
if (!job || job.status !== "running" || finishedAt < job.startedAt) return state;
return updateJobs(state, state.editor.generation.jobs.map((candidate) => candidate.id === jobId ? { ...candidate, status, finishedAt, error } : candidate));
}
function retainCandidateBudget(candidates: GenerationCandidate[]): GenerationCandidate[] {
const retained: GenerationCandidate[] = [];
let bytes = 0;
for (const candidate of candidates) {
const candidateBytes = candidateRetainedBytes(candidate);
if (retained.length > 0 && bytes + candidateBytes > maxCandidateSourceBytes) continue;
retained.push(candidate);
bytes += candidateBytes;
if (retained.length >= maxCandidates) break;
}
return retained;
}
function candidateRetainedBytes(candidate: GenerationCandidate): number {
return [candidate.source, candidate.inputImage, candidate.maskImage, candidate.blendMaskImage, candidate.inpaint?.inputImage, candidate.inpaint?.maskImage, candidate.inpaint?.editMaskImage, candidate.inpaint?.blendMaskImage]
.reduce((total, source) => total + (source?.length ?? 0) * 2, 0);
}
type LayerLocation = {
artboardId: ArtboardId;
layer: Layer;
@@ -382,11 +476,14 @@ function generationProvenance(candidate: GenerationCandidate, acceptance: Genera
scheduler: candidate.settings.scheduler,
width: candidate.settings.width,
height: candidate.settings.height,
batchSize: candidate.settings.batchSize,
refinePass: candidate.settings.refinePass,
refineStrength: candidate.settings.refineStrength,
},
inpaint: candidate.inpaint
? {
targetLayerId: candidate.inpaint.targetLayerId,
maskLayerId: candidate.inpaint.maskLayerId,
regionId: candidate.inpaint.regionId,
sourceAssetId: candidate.inpaint.sourceAssetId,
maskAssetId: candidate.inpaint.maskAssetId,
crop: {
@@ -406,6 +503,11 @@ function generationProvenance(candidate: GenerationCandidate, acceptance: Genera
maskFeather: candidate.inpaint.backend.maskFeather,
maskExpand: candidate.inpaint.backend.maskExpand,
cropPadding: candidate.inpaint.backend.cropPadding,
profile: candidate.settings.inpaint.profile,
structureControl: candidate.settings.inpaint.structureControl,
controlStrength: candidate.settings.inpaint.controlStrength,
controlModel: candidate.settings.inpaint.controlModel,
colorMatch: candidate.settings.inpaint.colorMatch,
},
}
: undefined,

View File

@@ -1,6 +1,8 @@
export const commandIds = {
projectOpen: "project.open",
documentAddArtboard: "document.addArtboard",
documentSetArtboardBounds: "document.setArtboardBounds",
documentResizeArtboard: "document.resizeArtboard",
documentRemoveArtboard: "document.removeArtboard",
documentSetArtboardVisible: "document.setArtboardVisible",
documentSetArtboardLocked: "document.setArtboardLocked",
@@ -10,29 +12,45 @@ export const commandIds = {
documentAddImageLayer: "document.addImageLayer",
documentAddRasterLayer: "document.addRasterLayer",
documentAddGroupLayer: "document.addGroupLayer",
documentAddAdjustmentLayer: "document.addAdjustmentLayer",
documentAddTextLayer: "document.addTextLayer",
documentSetTextLayer: "document.setTextLayer",
documentSetAdjustment: "document.setAdjustment",
documentMoveLayer: "document.moveLayer",
documentGroupLayers: "document.groupLayers",
documentUngroupLayer: "document.ungroupLayer",
documentRemoveLayer: "document.removeLayer",
documentSetLayerVisible: "document.setLayerVisible",
documentSetLayerLocked: "document.setLayerLocked",
documentSetLayerOpacity: "document.setLayerOpacity",
documentSetLayerSourceRect: "document.setLayerSourceRect",
documentDuplicateLayer: "document.duplicateLayer",
documentRenameLayer: "document.renameLayer",
documentSetLayerClippingMask: "document.setLayerClippingMask",
documentAddLayerMask: "document.addLayerMask",
documentApplyLayerMaskOperation: "document.applyLayerMaskOperation",
documentRemoveLayerMask: "document.removeLayerMask",
documentAddInpaintRegion: "document.addInpaintRegion",
documentApplyInpaintRegionMaskOperation: "document.applyInpaintRegionMaskOperation",
documentRemoveInpaintRegion: "document.removeInpaintRegion",
selectionSet: "selection.set",
selectionClear: "selection.clear",
selectionAddLayer: "selection.addLayer",
toolSetActive: "tool.setActive",
toolSetGenerateSettings: "tool.setGenerateSettings",
toolChooseGenerateIntent: "tool.chooseGenerateIntent",
toolSetBrushSettings: "tool.setBrushSettings",
toolSetFeatherSettings: "tool.setFeatherSettings",
toolSetChromaKeySettings: "tool.setChromaKeySettings",
toolSetMagicWandSettings: "tool.setMagicWandSettings",
toolSetBrushPreview: "tool.setBrushPreview",
toolSetBrushStrokePreview: "tool.setBrushStrokePreview",
toolBeginMaskShape: "tool.beginMaskShape",
toolAppendMaskShape: "tool.appendMaskShape",
toolClearMaskShape: "tool.clearMaskShape",
toolSetMaskViewMode: "tool.setMaskViewMode",
toolEnterMaskEdit: "tool.enterMaskEdit",
toolEnterInpaintRegionEdit: "tool.enterInpaintRegionEdit",
toolExitMaskEdit: "tool.exitMaskEdit",
toolEnterTemporaryPan: "tool.enterTemporaryPan",
toolExitTemporaryPan: "tool.exitTemporaryPan",
@@ -40,18 +58,23 @@ export const commandIds = {
generationSelectCandidate: "generation.selectCandidate",
generationSetCompareMode: "generation.setCompareMode",
generationRemoveCandidate: "generation.removeCandidate",
generationToggleCandidateFavorite: "generation.toggleCandidateFavorite",
generationClearCandidates: "generation.clearCandidates",
generationReuseCandidateSettings: "generation.reuseCandidateSettings",
generationApplyCandidateAsLayer: "generation.applyCandidateAsLayer",
generationReplaceCandidatePixels: "generation.replaceCandidatePixels",
generationStartJob: "generation.startJob",
generationUpdateJob: "generation.updateJob",
generationSucceedJob: "generation.succeedJob",
generationFailJob: "generation.failJob",
generationCancelJob: "generation.cancelJob",
generationLoadResources: "generation.loadResources",
generationSetResources: "generation.setResources",
generationFailResources: "generation.failResources",
transformBegin: "transform.begin",
transformUpdate: "transform.update",
transformSetBounds: "transform.setBounds",
transformSetRotation: "transform.setRotation",
transformEnd: "transform.end",
viewportPan: "viewport.pan",
viewportSetZoom: "viewport.setZoom",

View File

@@ -1,14 +1,21 @@
export type { Command, CommandContext } from "./command";
export { projectCommands, projectOpenCommand } from "./project";
export type { ProjectOpenPayload } from "./project";
export {
documentAddArtboardCommand,
documentAddAssetCommand,
documentAddGroupLayerCommand,
documentAddAdjustmentLayerCommand,
documentSetAdjustmentCommand,
documentAddTextLayerCommand,
documentSetTextLayerCommand,
documentAddImageLayerCommand,
documentAddLayerMaskCommand,
documentAddRasterLayerCommand,
documentApplyLayerMaskOperationCommand,
documentCommands,
documentGroupLayersCommand,
documentDuplicateLayerCommand,
documentMoveLayerCommand,
documentRemoveArtboardCommand,
documentRemoveLayerCommand,
@@ -20,6 +27,7 @@ export {
documentSetArtboardVisibleCommand,
documentSetLayerClippingMaskCommand,
documentSetLayerLockedCommand,
documentSetLayerOpacityCommand,
documentSetLayerVisibleCommand,
documentUpdateAssetSourceCommand,
documentUngroupLayerCommand,
@@ -28,11 +36,16 @@ export type {
DocumentAddArtboardPayload,
DocumentAddAssetPayload,
DocumentAddGroupLayerPayload,
DocumentAddAdjustmentLayerPayload,
DocumentSetAdjustmentPayload,
DocumentAddTextLayerPayload,
DocumentSetTextLayerPayload,
DocumentAddImageLayerPayload,
DocumentAddLayerMaskPayload,
DocumentAddRasterLayerPayload,
DocumentApplyLayerMaskOperationPayload,
DocumentGroupLayersPayload,
DocumentDuplicateLayerPayload,
DocumentMoveLayerPayload,
DocumentRemoveArtboardPayload,
DocumentRemoveLayerPayload,
@@ -44,6 +57,7 @@ export type {
DocumentSetArtboardVisiblePayload,
DocumentSetLayerClippingMaskPayload,
DocumentSetLayerLockedPayload,
DocumentSetLayerOpacityPayload,
DocumentSetLayerVisiblePayload,
DocumentUpdateAssetSourcePayload,
DocumentUngroupLayerPayload,
@@ -87,10 +101,12 @@ export type { CommandRegistry } from "./registry";
export { createCommandRegistry } from "./registry";
export { selectionAddLayerCommand, selectionClearCommand, selectionCommands, selectionSetCommand } from "./selection";
export type { SelectionAddLayerPayload, SelectionSetPayload } from "./selection";
export { toolCommands, toolEnterMaskEditCommand, toolEnterTemporaryPanCommand, toolExitMaskEditCommand, toolExitTemporaryPanCommand, toolSetActiveCommand, toolSetBrushPreviewCommand, toolSetBrushSettingsCommand, toolSetBrushStrokePreviewCommand, toolSetChromaKeySettingsCommand, toolSetGenerateSettingsCommand, toolSetMagicWandSettingsCommand, toolSetMaskViewModeCommand } from "./tool";
export { transformBeginCommand, transformCommands, transformEndCommand, transformSetBoundsCommand, transformUpdateCommand } from "./transform";
export type { TransformBeginPayload, TransformSetBoundsPayload, TransformUpdatePayload } from "./transform";
export type { ToolEnterMaskEditPayload, ToolSetActivePayload, ToolSetBrushPreviewPayload, ToolSetBrushSettingsPayload, ToolSetBrushStrokePreviewPayload, ToolSetChromaKeySettingsPayload, ToolSetGenerateSettingsPayload, ToolSetMagicWandSettingsPayload, ToolSetMaskViewModePayload } from "./tool";
export { toolChooseGenerateIntentCommand, toolCommands, toolEnterInpaintRegionEditCommand, toolEnterMaskEditCommand, toolEnterTemporaryPanCommand, toolExitMaskEditCommand, toolExitTemporaryPanCommand, toolSetActiveCommand, toolSetBrushPreviewCommand, toolSetBrushSettingsCommand, toolSetBrushStrokePreviewCommand, toolSetChromaKeySettingsCommand, toolSetGenerateSettingsCommand, toolSetMagicWandSettingsCommand, toolSetMaskViewModeCommand } from "./tool";
export { transformBeginCommand, transformCommands, transformEndCommand, transformSetBoundsCommand, transformSetRotationCommand, transformUpdateCommand } from "./transform";
export type { TransformBeginPayload, TransformSetBoundsPayload, TransformSetRotationPayload, TransformUpdatePayload } from "./transform";
export type { ToolChooseGenerateIntentPayload, ToolEnterInpaintRegionEditPayload, ToolEnterMaskEditPayload, ToolSetActivePayload, ToolSetBrushPreviewPayload, ToolSetBrushSettingsPayload, ToolSetBrushStrokePreviewPayload, ToolSetChromaKeySettingsPayload, ToolSetGenerateSettingsPayload, ToolSetMagicWandSettingsPayload, ToolSetMaskViewModePayload } from "./tool";
export { inpaintRegionCommands } from "./inpaint-region";
export type { DocumentAddInpaintRegionPayload, DocumentApplyInpaintRegionMaskOperationPayload, DocumentRemoveInpaintRegionPayload } from "./inpaint-region";
export {
viewportCommands,
viewportPanCommand,

View File

@@ -0,0 +1,32 @@
import { describe, expect, test } from "bun:test";
import { createInitialAppState } from "@editor/initial-state";
import { documentAddInpaintRegionCommand, documentApplyInpaintRegionMaskOperationCommand, documentRemoveInpaintRegionCommand } from "./inpaint-region";
describe("inpaint region commands", () => {
test("persists an AI edit mask independently from layer visibility", () => {
const state = stateWithTarget();
const added = documentAddInpaintRegionCommand.execute({ state }, {
region: { id: "region", name: "Target AI edit", targetLayerId: "target", maskAssetId: "mask", enabled: true },
maskAsset: { id: "mask", name: "Mask", mimeType: "image/png", source: "black", intrinsicSize: { w: 64, h: 32 } },
});
expect(added.document.inpaintRegions).toHaveLength(1);
expect(added.document.artboards[0]?.layers[0]).not.toHaveProperty("layerMask");
expect(added.document.artboards[0]?.layers[0]?.visible).toBe(true);
const painted = documentApplyInpaintRegionMaskOperationCommand.execute({ state: added }, { regionId: "region", source: "painted", operation: { type: "paint" } });
expect(painted.document.assets.find((asset) => asset.id === "mask")?.source).toBe("painted");
const removed = documentRemoveInpaintRegionCommand.execute({ state: painted }, { regionId: "region" });
expect(removed.document.inpaintRegions).toHaveLength(0);
expect(removed.document.assets.some((asset) => asset.id === "mask")).toBe(false);
});
});
function stateWithTarget() {
const state = createInitialAppState("Test");
state.document.assets.push({ id: "source", name: "Source", mimeType: "image/png", source: "pixels", intrinsicSize: { w: 64, h: 32 } });
state.document.artboards.push({ id: "board", name: "Board", bounds: { x: 0, y: 0, w: 64, h: 32 }, backgroundColor: "transparent", visible: true, locked: false, layers: [
{ id: "target", type: "raster", name: "Target", visible: true, locked: false, opacity: 1, assetId: "source", transform: { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 } },
] });
return state;
}

100
commands/inpaint-region.ts Normal file
View File

@@ -0,0 +1,100 @@
import type { Asset } from "@core/asset";
import type { InpaintRegion } from "@core/inpaint-region";
import type { InpaintRegionId } from "@core/id";
import type { Command } from "./command";
import { commandIds } from "./ids";
import type { LayerMaskOperation } from "./document";
import { findLayerLocation } from "./document-tree";
export type DocumentAddInpaintRegionPayload = {
region: InpaintRegion;
maskAsset: Asset;
};
export type DocumentApplyInpaintRegionMaskOperationPayload = {
regionId: InpaintRegionId;
source: string;
mimeType?: string;
operation: LayerMaskOperation;
};
export type DocumentRemoveInpaintRegionPayload = {
regionId: InpaintRegionId;
};
export const documentAddInpaintRegionCommand: Command<DocumentAddInpaintRegionPayload> = {
id: commandIds.documentAddInpaintRegion,
name: "Add inpaint region",
execute({ state }, payload) {
const target = findLayerLocation(state.document, payload.region.targetLayerId)?.layer;
if (!target || (target.type !== "image" && target.type !== "raster")) return state;
const targetAsset = state.document.assets.find((asset) => asset.id === target.assetId);
if (!targetAsset || state.document.inpaintRegions.some((region) => region.id === payload.region.id)) return state;
if (state.document.assets.some((asset) => asset.id === payload.maskAsset.id)) return state;
if (payload.region.maskAssetId !== payload.maskAsset.id || !payload.region.name.trim()) return state;
if (Math.round(payload.maskAsset.intrinsicSize.w) !== Math.round(targetAsset.intrinsicSize.w) || Math.round(payload.maskAsset.intrinsicSize.h) !== Math.round(targetAsset.intrinsicSize.h)) return state;
return {
...state,
document: {
...state.document,
assets: [...state.document.assets, payload.maskAsset],
inpaintRegions: [...state.document.inpaintRegions, { ...payload.region, name: payload.region.name.trim() }],
},
};
},
};
export const documentApplyInpaintRegionMaskOperationCommand: Command<DocumentApplyInpaintRegionMaskOperationPayload> = {
id: commandIds.documentApplyInpaintRegionMaskOperation,
name: "Apply inpaint region mask operation",
execute({ state }, payload) {
const region = state.document.inpaintRegions.find((candidate) => candidate.id === payload.regionId);
if (!region || !payload.source.trim()) return state;
return {
...state,
document: {
...state.document,
assets: state.document.assets.map((asset) => asset.id === region.maskAssetId ? { ...asset, source: payload.source, mimeType: payload.mimeType ?? asset.mimeType } : asset),
},
editor: {
...state.editor,
brushStrokePreview: state.editor.brushStrokePreview?.assetId === region.maskAssetId ? undefined : state.editor.brushStrokePreview,
},
};
},
};
export const documentRemoveInpaintRegionCommand: Command<DocumentRemoveInpaintRegionPayload> = {
id: commandIds.documentRemoveInpaintRegion,
name: "Remove inpaint region",
execute({ state }, payload) {
const region = state.document.inpaintRegions.find((candidate) => candidate.id === payload.regionId);
if (!region) return state;
const inpaintRegions = state.document.inpaintRegions.filter((candidate) => candidate.id !== payload.regionId);
const maskStillUsed = inpaintRegions.some((candidate) => candidate.maskAssetId === region.maskAssetId);
const exitingEdit = state.editor.maskEdit?.inpaintRegionId === region.id;
const contextualTool = state.editor.tools.activeTool === "semanticSelect" || state.editor.tools.activeTool === "maskLasso" || state.editor.tools.activeTool === "maskRectangle";
return {
...state,
document: {
...state.document,
inpaintRegions,
assets: maskStillUsed ? state.document.assets : state.document.assets.filter((asset) => asset.id !== region.maskAssetId),
},
editor: {
...state.editor,
maskEdit: exitingEdit ? undefined : state.editor.maskEdit,
brushStrokePreview: state.editor.brushStrokePreview?.assetId === region.maskAssetId ? undefined : state.editor.brushStrokePreview,
maskShapeSession: exitingEdit ? undefined : state.editor.maskShapeSession,
tools: exitingEdit && contextualTool ? { ...state.editor.tools, activeTool: "select", interactionMode: { type: "tool", tool: "select" } } : state.editor.tools,
},
};
},
};
export const inpaintRegionCommands = [
documentAddInpaintRegionCommand,
documentApplyInpaintRegionMaskOperationCommand,
documentRemoveInpaintRegionCommand,
] satisfies Command<unknown>[];

View File

@@ -3,11 +3,14 @@ import type {
DocumentAddArtboardPayload,
DocumentAddAssetPayload,
DocumentAddGroupLayerPayload,
DocumentAddAdjustmentLayerPayload,
DocumentSetAdjustmentPayload,
DocumentAddImageLayerPayload,
DocumentAddLayerMaskPayload,
DocumentAddRasterLayerPayload,
DocumentApplyLayerMaskOperationPayload,
DocumentGroupLayersPayload,
DocumentDuplicateLayerPayload,
DocumentMoveLayerPayload,
DocumentRemoveArtboardPayload,
DocumentRemoveLayerPayload,
@@ -15,24 +18,33 @@ import type {
DocumentRenameArtboardPayload,
DocumentRenameLayerPayload,
DocumentSetArtboardBoundsPayload,
DocumentResizeArtboardPayload,
DocumentSetArtboardLockedPayload,
DocumentSetArtboardVisiblePayload,
DocumentSetLayerClippingMaskPayload,
DocumentSetLayerLockedPayload,
DocumentSetLayerOpacityPayload,
DocumentSetLayerSourceRectPayload,
DocumentSetLayerVisiblePayload,
DocumentUpdateAssetSourcePayload,
DocumentUngroupLayerPayload,
DocumentAddTextLayerPayload,
DocumentSetTextLayerPayload,
} from "./document";
import type {
GenerationAddCandidatePayload,
GenerationApplyCandidateAsLayerPayload,
GenerationRemoveCandidatePayload,
GenerationToggleCandidateFavoritePayload,
GenerationReuseCandidateSettingsPayload,
GenerationReplaceCandidatePixelsPayload,
GenerationSelectCandidatePayload,
GenerationSetCompareModePayload,
GenerationStartJobPayload,
GenerationUpdateJobPayload,
GenerationSucceedJobPayload,
GenerationFailJobPayload,
GenerationCancelJobPayload,
GenerationSetResourcesPayload,
GenerationFailResourcesPayload,
} from "./generation";
@@ -42,10 +54,12 @@ import type {
CommandPaletteSetSelectedIndexPayload,
} from "./palette";
import type { SelectionAddLayerPayload, SelectionSetPayload } from "./selection";
import type { ToolEnterMaskEditPayload, ToolSetActivePayload, ToolSetBrushPreviewPayload, ToolSetBrushSettingsPayload, ToolSetBrushStrokePreviewPayload, ToolSetChromaKeySettingsPayload, ToolSetGenerateSettingsPayload, ToolSetMagicWandSettingsPayload, ToolSetMaskViewModePayload } from "./tool";
import type { TransformBeginPayload, TransformSetBoundsPayload, TransformUpdatePayload } from "./transform";
import type { ToolAppendMaskShapePayload, ToolBeginMaskShapePayload, ToolChooseGenerateIntentPayload, ToolEnterInpaintRegionEditPayload, ToolEnterMaskEditPayload, ToolSetActivePayload, ToolSetBrushPreviewPayload, ToolSetBrushSettingsPayload, ToolSetBrushStrokePreviewPayload, ToolSetChromaKeySettingsPayload, ToolSetFeatherSettingsPayload, ToolSetGenerateSettingsPayload, ToolSetMagicWandSettingsPayload, ToolSetMaskViewModePayload } from "./tool";
import type { TransformBeginPayload, TransformSetBoundsPayload, TransformSetRotationPayload, TransformUpdatePayload } from "./transform";
import type { WorkspaceSetPanelPayload } from "./workspace";
import type { EditorSetPointerSessionPayload } from "./editor";
import type { ProjectOpenPayload } from "./project";
import type { DocumentAddInpaintRegionPayload, DocumentApplyInpaintRegionMaskOperationPayload, DocumentRemoveInpaintRegionPayload } from "./inpaint-region";
import type {
ViewportFitArtboardPayload,
ViewportPanPayload,
@@ -55,8 +69,10 @@ import type {
} from "./viewport";
export type CommandPayloads = {
[commandIds.projectOpen]: ProjectOpenPayload;
[commandIds.documentAddArtboard]: DocumentAddArtboardPayload;
[commandIds.documentSetArtboardBounds]: DocumentSetArtboardBoundsPayload;
[commandIds.documentResizeArtboard]: DocumentResizeArtboardPayload;
[commandIds.documentRemoveArtboard]: DocumentRemoveArtboardPayload;
[commandIds.documentSetArtboardVisible]: DocumentSetArtboardVisiblePayload;
[commandIds.documentSetArtboardLocked]: DocumentSetArtboardLockedPayload;
@@ -66,15 +82,25 @@ export type CommandPayloads = {
[commandIds.documentAddImageLayer]: DocumentAddImageLayerPayload;
[commandIds.documentAddRasterLayer]: DocumentAddRasterLayerPayload;
[commandIds.documentAddGroupLayer]: DocumentAddGroupLayerPayload;
[commandIds.documentAddAdjustmentLayer]: DocumentAddAdjustmentLayerPayload;
[commandIds.documentSetAdjustment]: DocumentSetAdjustmentPayload;
[commandIds.documentAddTextLayer]: DocumentAddTextLayerPayload;
[commandIds.documentSetTextLayer]: DocumentSetTextLayerPayload;
[commandIds.documentAddLayerMask]: DocumentAddLayerMaskPayload;
[commandIds.documentApplyLayerMaskOperation]: DocumentApplyLayerMaskOperationPayload;
[commandIds.documentRemoveLayerMask]: DocumentRemoveLayerMaskPayload;
[commandIds.documentAddInpaintRegion]: DocumentAddInpaintRegionPayload;
[commandIds.documentApplyInpaintRegionMaskOperation]: DocumentApplyInpaintRegionMaskOperationPayload;
[commandIds.documentRemoveInpaintRegion]: DocumentRemoveInpaintRegionPayload;
[commandIds.documentMoveLayer]: DocumentMoveLayerPayload;
[commandIds.documentGroupLayers]: DocumentGroupLayersPayload;
[commandIds.documentUngroupLayer]: DocumentUngroupLayerPayload;
[commandIds.documentRemoveLayer]: DocumentRemoveLayerPayload;
[commandIds.documentSetLayerVisible]: DocumentSetLayerVisiblePayload;
[commandIds.documentSetLayerLocked]: DocumentSetLayerLockedPayload;
[commandIds.documentSetLayerOpacity]: DocumentSetLayerOpacityPayload;
[commandIds.documentSetLayerSourceRect]: DocumentSetLayerSourceRectPayload;
[commandIds.documentDuplicateLayer]: DocumentDuplicateLayerPayload;
[commandIds.documentRenameLayer]: DocumentRenameLayerPayload;
[commandIds.documentSetLayerClippingMask]: DocumentSetLayerClippingMaskPayload;
[commandIds.selectionSet]: SelectionSetPayload;
@@ -82,13 +108,19 @@ export type CommandPayloads = {
[commandIds.selectionAddLayer]: SelectionAddLayerPayload;
[commandIds.toolSetActive]: ToolSetActivePayload;
[commandIds.toolSetGenerateSettings]: ToolSetGenerateSettingsPayload;
[commandIds.toolChooseGenerateIntent]: ToolChooseGenerateIntentPayload;
[commandIds.toolSetBrushSettings]: ToolSetBrushSettingsPayload;
[commandIds.toolSetFeatherSettings]: ToolSetFeatherSettingsPayload;
[commandIds.toolSetChromaKeySettings]: ToolSetChromaKeySettingsPayload;
[commandIds.toolSetMagicWandSettings]: ToolSetMagicWandSettingsPayload;
[commandIds.toolSetBrushPreview]: ToolSetBrushPreviewPayload;
[commandIds.toolSetBrushStrokePreview]: ToolSetBrushStrokePreviewPayload;
[commandIds.toolBeginMaskShape]: ToolBeginMaskShapePayload;
[commandIds.toolAppendMaskShape]: ToolAppendMaskShapePayload;
[commandIds.toolClearMaskShape]: void;
[commandIds.toolSetMaskViewMode]: ToolSetMaskViewModePayload;
[commandIds.toolEnterMaskEdit]: ToolEnterMaskEditPayload;
[commandIds.toolEnterInpaintRegionEdit]: ToolEnterInpaintRegionEditPayload;
[commandIds.toolExitMaskEdit]: void;
[commandIds.toolEnterTemporaryPan]: void;
[commandIds.toolExitTemporaryPan]: void;
@@ -96,18 +128,23 @@ export type CommandPayloads = {
[commandIds.generationSelectCandidate]: GenerationSelectCandidatePayload;
[commandIds.generationSetCompareMode]: GenerationSetCompareModePayload;
[commandIds.generationRemoveCandidate]: GenerationRemoveCandidatePayload;
[commandIds.generationToggleCandidateFavorite]: GenerationToggleCandidateFavoritePayload;
[commandIds.generationClearCandidates]: void;
[commandIds.generationReuseCandidateSettings]: GenerationReuseCandidateSettingsPayload;
[commandIds.generationApplyCandidateAsLayer]: GenerationApplyCandidateAsLayerPayload;
[commandIds.generationReplaceCandidatePixels]: GenerationReplaceCandidatePixelsPayload;
[commandIds.generationStartJob]: GenerationStartJobPayload;
[commandIds.generationUpdateJob]: GenerationUpdateJobPayload;
[commandIds.generationSucceedJob]: GenerationSucceedJobPayload;
[commandIds.generationFailJob]: GenerationFailJobPayload;
[commandIds.generationCancelJob]: GenerationCancelJobPayload;
[commandIds.generationLoadResources]: void;
[commandIds.generationSetResources]: GenerationSetResourcesPayload;
[commandIds.generationFailResources]: GenerationFailResourcesPayload;
[commandIds.transformBegin]: TransformBeginPayload;
[commandIds.transformUpdate]: TransformUpdatePayload;
[commandIds.transformSetBounds]: TransformSetBoundsPayload;
[commandIds.transformSetRotation]: TransformSetRotationPayload;
[commandIds.transformEnd]: void;
[commandIds.viewportPan]: ViewportPanPayload;
[commandIds.viewportSetZoom]: ViewportSetZoomPayload;

31
commands/project.test.ts Normal file
View File

@@ -0,0 +1,31 @@
import { describe, expect, test } from "bun:test";
import { createInitialAppState } from "@editor/initial-state";
import { createAppStore } from "@editor/store";
import { createCommandRegistry } from "./registry";
import { documentAddArtboardCommand, documentRenameArtboardCommand } from "./document";
import { projectOpenCommand } from "./project";
import { commandIds } from "./ids";
describe("open project command", () => {
test("replaces the document and resets transient sessions and history", () => {
const app = createTestApp("Original");
const replacement = createTestApp("Opened").store.getState().document;
const originalArtboardId = app.store.getState().document.artboards[0]!.id;
app.store.dispatch(commandIds.documentRenameArtboard, { id: originalArtboardId, name: "Changed" });
expect(app.store.getState().history.past).toHaveLength(2);
app.store.dispatch(commandIds.projectOpen, { document: replacement });
const state = app.store.getState();
expect(state.document).toBe(replacement);
expect(state.editor.selection).toEqual({ artboardId: replacement.artboards[0]!.id, layerIds: [] });
expect(state.editor.generation.candidates).toEqual([]);
expect(state.history).toEqual({ past: [], future: [] });
});
});
function createTestApp(name: string) {
const store = createAppStore(createInitialAppState(name), createCommandRegistry([projectOpenCommand, documentAddArtboardCommand, documentRenameArtboardCommand]));
store.dispatch(commandIds.documentAddArtboard, { id: `${name}-artboard`, name: "Board", bounds: { x: 0, y: 0, w: 100, h: 100 } });
return { store };
}

37
commands/project.ts Normal file
View File

@@ -0,0 +1,37 @@
import type { ImageDocument } from "@core/document";
import { initialEditorState } from "@editor/initial-state";
import type { Command } from "./command";
import { commandIds } from "./ids";
export type ProjectOpenPayload = { document: ImageDocument };
export const projectOpenCommand: Command<ProjectOpenPayload> = {
id: commandIds.projectOpen,
name: "Open project",
history: { mode: "ignore" },
execute({ state }, payload) {
const firstArtboard = payload.document.artboards[0];
const viewport = state.editor.viewport;
const padding = 48;
const availableWidth = Math.max(0, viewport.size.w - padding * 2);
const availableHeight = Math.max(0, viewport.size.h - padding * 2);
const canFit = Boolean(firstArtboard && availableWidth > 0 && availableHeight > 0 && firstArtboard.bounds.w > 0 && firstArtboard.bounds.h > 0);
return {
document: payload.document,
editor: {
...initialEditorState,
viewport: {
...viewport,
center: firstArtboard
? { x: firstArtboard.bounds.x + firstArtboard.bounds.w / 2, y: firstArtboard.bounds.y + firstArtboard.bounds.h / 2 }
: viewport.center,
zoom: canFit && firstArtboard ? Math.max(0.01, Math.min(availableWidth / firstArtboard.bounds.w, availableHeight / firstArtboard.bounds.h)) : viewport.zoom,
},
selection: { artboardId: firstArtboard?.id, layerIds: [] },
},
history: { past: [], future: [] },
};
},
};
export const projectCommands = [projectOpenCommand] satisfies Command<unknown>[];

View File

@@ -0,0 +1,23 @@
import { describe, expect, test } from "bun:test";
import { createInitialAppState } from "@editor/initial-state";
import { documentAddArtboardCommand, documentAddTextLayerCommand, documentGroupLayersCommand, documentSetTextLayerCommand } from "./document";
import type { TextLayer } from "@core/text-layer";
const text: TextLayer = { id: "text", type: "text", name: "Text", visible: true, locked: false, opacity: 1, transform: { position: { x: 5, y: 6 }, scale: { x: 1, y: 1 }, rotation: 0 }, content: "Hello", style: { fontFamily: "Arial", fontSize: 24, fontWeight: 400, fontStyle: "normal", color: "#112233", alignment: "left", lineHeight: 1.2 } };
describe("text layer commands", () => {
test("creates, selects and edits authoritative text", () => {
const base = documentAddArtboardCommand.execute({ state: createInitialAppState("Test") }, { id: "board", name: "Board", bounds: { x: 0, y: 0, w: 100, h: 100 } });
const added = documentAddTextLayerCommand.execute({ state: base }, { artboardId: "board", layer: text });
expect(added.editor.selection.layerIds).toEqual(["text"]);
const edited = documentSetTextLayerCommand.execute({ state: added }, { layerId: "text", content: "Changed", style: { ...text.style, fontWeight: 700 } });
expect(edited.document.artboards[0]?.layers[0]).toMatchObject({ type: "text", content: "Changed", style: { fontWeight: 700 } });
});
test("can be grouped as a real tree node", () => {
const base = documentAddArtboardCommand.execute({ state: createInitialAppState("Test") }, { id: "board", name: "Board", bounds: { x: 0, y: 0, w: 100, h: 100 } });
const added = documentAddTextLayerCommand.execute({ state: base }, { artboardId: "board", layer: text });
const grouped = documentGroupLayersCommand.execute({ state: added }, { artboardId: "board", layerIds: ["text"], group: { id: "group", type: "group", name: "Group", visible: true, locked: false, opacity: 1, transform: { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 }, children: [] } });
expect(grouped.document.artboards[0]?.layers[0]).toMatchObject({ type: "group", children: [{ id: "text", type: "text" }] });
});
});

View File

@@ -1,22 +1,38 @@
import { describe, expect, test } from "bun:test";
import { createInitialAppState } from "@editor/initial-state";
import { toolEnterMaskEditCommand, toolEnterTemporaryPanCommand, toolExitMaskEditCommand, toolExitTemporaryPanCommand, toolSetActiveCommand, toolSetBrushPreviewCommand, toolSetBrushSettingsCommand, toolSetBrushStrokePreviewCommand, toolSetChromaKeySettingsCommand, toolSetGenerateSettingsCommand, toolSetMaskViewModeCommand } from "./tool";
import { initialToolState } from "@editor/tools";
import { toolAppendMaskShapeCommand, toolBeginMaskShapeCommand, toolChooseGenerateIntentCommand, toolEnterMaskEditCommand, toolEnterTemporaryPanCommand, toolExitMaskEditCommand, toolExitTemporaryPanCommand, toolSetActiveCommand, toolSetBrushPreviewCommand, toolSetBrushSettingsCommand, toolSetBrushStrokePreviewCommand, toolSetChromaKeySettingsCommand, toolSetFeatherSettingsCommand, toolSetGenerateSettingsCommand, toolSetMaskViewModeCommand } from "./tool";
const defaultBrush = { color: "#111827", size: 8, hardness: 100 };
const defaultBrush = { color: "#111827", size: 8, hardness: 100, opacity: 100, flow: 100, smoothing: 20, pressureSize: true };
const defaultFeather = { size: 96, radius: 16, strength: 65, smoothing: 25, pressureSize: true };
const defaultChromaKey = { color: "#00ff00", tolerance: 32, softness: 24, feather: 0, choke: 0, despeckle: 0, spill: 50 };
const defaultMagicWand = { tolerance: 32, feather: 0, choke: 0, despeckle: 0, contiguous: true, mode: "replace" as const };
const defaultGenerate = { architecture: "sdxl" as const, mode: "text-to-image" as const, model: "auto" as const, textEncoder: "auto", vae: "auto", prompt: "", negativePrompt: "", strength: 75, steps: 30, cfg: 7, seed: -1, sampler: "euler", scheduler: "normal", width: 1024, height: 1024, outpaint: { left: 128, top: 128, right: 128, bottom: 128, feathering: 32 }, inpaint: { maskedAreaOnly: true, cropPadding: 96, maskPolarity: "hidden" as const, maskedContent: "neutral" as const, growMaskBy: 6, maskExpand: 0, maskFeather: 0, maskBlur: 0, maskDespeckle: 0 } };
const defaultGenerate = initialToolState.generate;
describe("tool commands", () => {
test("sets active tool", () => {
const next = toolSetActiveCommand.execute({ state: createInitialAppState("Test") }, { tool: "brush" });
expect(next.editor.tools).toEqual({ activeTool: "brush", interactionMode: { type: "tool", tool: "brush" }, brush: defaultBrush, generate: defaultGenerate, chromaKey: defaultChromaKey, magicWand: defaultMagicWand });
expect(next.editor.tools).toEqual({ activeTool: "brush", interactionMode: { type: "tool", tool: "brush" }, brush: defaultBrush, feather: defaultFeather, generate: defaultGenerate, chromaKey: defaultChromaKey, magicWand: defaultMagicWand });
});
test("selecting a persistent tool closes an open operation", () => {
const state = createInitialAppState("Test");
state.editor.workspace = { panel: "chromaKey" };
const next = toolSetActiveCommand.execute({ state }, { tool: "brush" });
expect(next.editor.tools.activeTool).toBe("brush");
expect(next.editor.workspace.panel).toBe("none");
});
test("sets brush settings", () => {
const next = toolSetBrushSettingsCommand.execute({ state: createInitialAppState("Test") }, { color: "#ff0000", size: 24, hardness: 50 });
expect(next.editor.tools.brush).toEqual({ color: "#ff0000", size: 24, hardness: 50 });
expect(next.editor.tools.brush).toEqual({ ...defaultBrush, color: "#ff0000", size: 24, hardness: 50 });
});
test("sets and clamps feather settings", () => {
const next = toolSetFeatherSettingsCommand.execute({ state: createInitialAppState("Test") }, { size: 800, radius: 0, strength: 45 });
expect(next.editor.tools.feather).toEqual({ ...defaultFeather, size: 400, radius: 1, strength: 45 });
});
test("sets chroma key settings", () => {
@@ -35,6 +51,11 @@ describe("tool commands", () => {
expect(next.editor.tools.generate.inpaint).toEqual({ ...defaultGenerate.inpaint, cropPadding: 2048, growMaskBy: 0, maskExpand: -256, maskBlur: 256, maskPolarity: "revealed", maskedContent: "original" });
});
test("applies outcome-oriented inpaint profile defaults", () => {
const next = toolSetGenerateSettingsCommand.execute({ state: createInitialAppState("Test") }, { mode: "inpaint", inpaint: { profile: "material" } });
expect(next.editor.tools.generate).toMatchObject({ strength: 65, inpaint: { profile: "material", maskedContent: "edges", structureControl: "canny", controlStrength: 0.55, colorMatch: false } });
});
test("applies architecture defaults and filters unsupported modes", () => {
const initial = toolSetGenerateSettingsCommand.execute({ state: createInitialAppState("Test") }, { mode: "inpaint" });
const next = toolSetGenerateSettingsCommand.execute({ state: initial }, { architecture: "z-image-turbo" });
@@ -52,6 +73,20 @@ describe("tool commands", () => {
});
});
test("maps user intents to supported generation modes and a compatible architecture", () => {
const zImage = toolSetGenerateSettingsCommand.execute({ state: createInitialAppState("Test") }, { architecture: "z-image" });
const replace = toolChooseGenerateIntentCommand.execute({ state: zImage }, { intent: "replace" });
const variations = toolChooseGenerateIntentCommand.execute({ state: replace }, { intent: "variations" });
expect(replace.editor.tools.generate).toMatchObject({ architecture: "sdxl", mode: "inpaint" });
expect(variations.editor.tools.generate).toMatchObject({ architecture: "sdxl", mode: "image-to-image" });
});
test("maps object removal to the dedicated inpaint profile", () => {
const removed = toolChooseGenerateIntentCommand.execute({ state: createInitialAppState("Test") }, { intent: "remove" });
expect(removed.editor.tools.generate).toMatchObject({ architecture: "sdxl", mode: "inpaint", strength: 100, inpaint: { profile: "remove", maskedContent: "neutral", maskExpand: 12 } });
});
test("sets and clears brush preview", () => {
const showing = toolSetBrushPreviewCommand.execute({ state: createInitialAppState("Test") }, { position: { x: 10, y: 20 } });
const cleared = toolSetBrushPreviewCommand.execute({ state: showing }, undefined);
@@ -68,11 +103,25 @@ describe("tool commands", () => {
expect(cleared.editor.brushStrokePreview).toBeUndefined();
});
test("keeps rectangle mask gestures as two command-owned corner points", () => {
const state = createInitialAppState("Test");
state.editor = {
...state.editor,
tools: { ...state.editor.tools, activeTool: "maskRectangle", interactionMode: { type: "tool", tool: "maskRectangle" } },
maskEdit: { kind: "inpaintRegion", targetLayerId: "target", inpaintRegionId: "region", maskAssetId: "mask" },
};
const begun = toolBeginMaskShapeCommand.execute({ state }, { point: { x: 10, y: 20 }, mode: "add" });
const moved = toolAppendMaskShapeCommand.execute({ state: begun }, { point: { x: 40, y: 60 } });
const movedAgain = toolAppendMaskShapeCommand.execute({ state: moved }, { point: { x: 50, y: 70 } });
expect(movedAgain.editor.maskShapeSession).toEqual({ shape: "rectangle", mode: "add", points: [{ x: 10, y: 20 }, { x: 50, y: 70 }] });
});
test("enters and exits mask edit", () => {
const editing = toolEnterMaskEditCommand.execute({ state: stateWithMask() }, { targetLayerId: "target", maskLayerId: "mask" });
const exited = toolExitMaskEditCommand.execute({ state: editing }, undefined);
expect(editing.editor.maskEdit).toEqual({ targetLayerId: "target", maskLayerId: "mask" });
expect(editing.editor.maskEdit).toEqual({ kind: "layerMask", targetLayerId: "target", maskLayerId: "mask", maskAssetId: "mask-asset" });
expect(editing.editor.selection).toEqual({ artboardId: "a1", layerIds: ["target"] });
expect(editing.editor.tools.activeTool).toBe("brush");
expect(exited.editor.maskEdit).toBeUndefined();
@@ -83,7 +132,7 @@ describe("tool commands", () => {
const alpha = toolSetMaskViewModeCommand.execute({ state: editing }, { mode: "alpha" });
const ignored = toolSetMaskViewModeCommand.execute({ state: createInitialAppState("Test") }, { mode: "blackWhite" });
expect(alpha.editor.maskEdit).toEqual({ targetLayerId: "target", maskLayerId: "mask", viewMode: "alpha" });
expect(alpha.editor.maskEdit).toEqual({ kind: "layerMask", targetLayerId: "target", maskLayerId: "mask", maskAssetId: "mask-asset", viewMode: "alpha" });
expect(ignored.editor.maskEdit).toBeUndefined();
});
@@ -92,7 +141,7 @@ describe("tool commands", () => {
const panning = toolEnterTemporaryPanCommand.execute({ state: initial }, undefined);
const restored = toolExitTemporaryPanCommand.execute({ state: panning }, undefined);
expect(panning.editor.tools).toEqual({ activeTool: "select", interactionMode: { type: "temporary-pan", previousTool: "select" }, brush: defaultBrush, generate: defaultGenerate, chromaKey: defaultChromaKey, magicWand: defaultMagicWand });
expect(panning.editor.tools).toEqual({ activeTool: "select", interactionMode: { type: "temporary-pan", previousTool: "select" }, brush: defaultBrush, feather: defaultFeather, generate: defaultGenerate, chromaKey: defaultChromaKey, magicWand: defaultMagicWand });
expect(restored.editor.tools).toEqual(initial.editor.tools);
});
});

View File

@@ -1,11 +1,11 @@
import type { ImageDocument } from "@core/document";
import type { Vec2D } from "@core/geometry";
import type { LayerId, ArtboardId, AssetId } from "@core/id";
import type { LayerId, ArtboardId, AssetId, InpaintRegionId } from "@core/id";
import type { Layer } from "@core/layer";
import { getLayerMask } from "@core/layer-mask-utils";
import type { MaskViewMode } from "@editor/state";
import { generateArchitectureDefaults } from "@editor/tools";
import type { BrushSettings, ChromaKeySettings, GenerateSettings, MagicWandSettings, ToolId } from "@editor/tools";
import { generateArchitectureDefaults, inpaintProfileDefaults } from "@editor/tools";
import type { BrushSettings, ChromaKeySettings, FeatherSettings, GenerateIntent, GenerateSettings, MagicWandSettings, ToolId } from "@editor/tools";
import type { Command } from "./command";
import { commandIds } from "./ids";
@@ -14,8 +14,13 @@ export type ToolSetActivePayload = {
};
export type ToolSetBrushSettingsPayload = Partial<BrushSettings>;
export type ToolSetFeatherSettingsPayload = Partial<FeatherSettings>;
export type ToolSetGenerateSettingsPayload = Partial<GenerateSettings>;
export type ToolSetGenerateSettingsPayload = Omit<Partial<GenerateSettings>, "inpaint" | "outpaint"> & {
inpaint?: Partial<GenerateSettings["inpaint"]>;
outpaint?: Partial<GenerateSettings["outpaint"]>;
};
export type ToolChooseGenerateIntentPayload = { intent: GenerateIntent };
export type ToolSetChromaKeySettingsPayload = Partial<ChromaKeySettings>;
@@ -28,24 +33,31 @@ export type ToolSetBrushStrokePreviewPayload =
layerId: LayerId;
assetId: AssetId;
source: string;
pendingTargetLayerId?: LayerId;
intrinsicSize?: { w: number; h: number };
}
| undefined;
export type ToolSetMaskViewModePayload = {
mode: MaskViewMode;
};
export type ToolBeginMaskShapePayload = { point: Vec2D; mode: "replace" | "add" | "subtract" };
export type ToolAppendMaskShapePayload = { point: Vec2D };
export type ToolEnterMaskEditPayload = {
targetLayerId: LayerId;
maskLayerId: LayerId;
};
export type ToolEnterInpaintRegionEditPayload = {
targetLayerId: LayerId;
regionId: InpaintRegionId;
};
export const toolSetActiveCommand: Command<ToolSetActivePayload> = {
id: commandIds.toolSetActive,
name: "Set active tool",
execute({ state }, payload) {
const previousNonGenerateTool = payload.tool === "generate" ? state.editor.workspace.previousNonGenerateTool : payload.tool;
const panel = payload.tool === "generate" ? "generate" : state.editor.workspace.panel === "generate" ? "none" : state.editor.workspace.panel;
return {
...state,
editor: {
@@ -57,7 +69,10 @@ export const toolSetActiveCommand: Command<ToolSetActivePayload> = {
},
brushPreview: undefined,
brushStrokePreview: undefined,
workspace: { panel, previousNonGenerateTool },
maskShapeSession: undefined,
workspace: state.editor.workspace.panel === "generate" || state.editor.workspace.panel === "chromaKey"
? { panel: "none" }
: state.editor.workspace,
},
};
},
@@ -73,6 +88,8 @@ export const toolSetGenerateSettingsCommand: Command<ToolSetGenerateSettingsPayl
const defaults = generateArchitectureDefaults[architecture];
const requestedMode = payload.mode ?? current.mode;
const mode = defaults.supportedModes.includes(requestedMode) ? requestedMode : defaults.supportedModes[0] ?? "text-to-image";
const requestedProfile = payload.inpaint?.profile ?? current.inpaint.profile;
const profileDefaults = requestedProfile !== current.inpaint.profile && requestedProfile !== "custom" ? inpaintProfileDefaults[requestedProfile] : undefined;
return {
...state,
editor: {
@@ -87,7 +104,7 @@ export const toolSetGenerateSettingsCommand: Command<ToolSetGenerateSettingsPayl
vae: payload.vae ?? (architectureChanged ? defaults.vae : current.vae),
prompt: payload.prompt ?? current.prompt,
negativePrompt: payload.negativePrompt ?? current.negativePrompt,
strength: clampNumber(payload.strength ?? current.strength, 0, 100),
strength: clampNumber(payload.strength ?? profileDefaults?.strength ?? current.strength, 0, 100),
steps: Math.round(clampNumber(payload.steps ?? (architectureChanged ? defaults.steps : current.steps), 1, 150)),
cfg: clampNumber(payload.cfg ?? (architectureChanged ? defaults.cfg : current.cfg), 0, 30),
seed: Math.round(clampNumber(payload.seed ?? current.seed, -1, Number.MAX_SAFE_INTEGER)),
@@ -95,6 +112,9 @@ export const toolSetGenerateSettingsCommand: Command<ToolSetGenerateSettingsPayl
scheduler: payload.scheduler ?? (architectureChanged ? defaults.scheduler : current.scheduler),
width: Math.round(clampNumber(payload.width ?? current.width, 64, 4096)),
height: Math.round(clampNumber(payload.height ?? current.height, 64, 4096)),
batchSize: Math.round(clampNumber(payload.batchSize ?? current.batchSize, 1, 8)),
refinePass: payload.refinePass ?? profileDefaults?.refinePass ?? current.refinePass,
refineStrength: clampNumber(payload.refineStrength ?? profileDefaults?.refineStrength ?? current.refineStrength, 0, 100),
outpaint: {
left: Math.round(clampNumber(payload.outpaint?.left ?? current.outpaint.left, 0, 2048)),
top: Math.round(clampNumber(payload.outpaint?.top ?? current.outpaint.top, 0, 2048)),
@@ -103,15 +123,20 @@ export const toolSetGenerateSettingsCommand: Command<ToolSetGenerateSettingsPayl
feathering: Math.round(clampNumber(payload.outpaint?.feathering ?? current.outpaint.feathering, 0, 512)),
},
inpaint: {
profile: requestedProfile,
maskedAreaOnly: payload.inpaint?.maskedAreaOnly ?? current.inpaint.maskedAreaOnly,
cropPadding: Math.round(clampNumber(payload.inpaint?.cropPadding ?? current.inpaint.cropPadding, 0, 2048)),
cropPadding: Math.round(clampNumber(payload.inpaint?.cropPadding ?? profileDefaults?.inpaint.cropPadding ?? current.inpaint.cropPadding, 0, 2048)),
maskPolarity: payload.inpaint?.maskPolarity ?? current.inpaint.maskPolarity,
maskedContent: payload.inpaint?.maskedContent ?? current.inpaint.maskedContent,
maskedContent: payload.inpaint?.maskedContent ?? profileDefaults?.inpaint.maskedContent ?? current.inpaint.maskedContent,
growMaskBy: Math.round(clampNumber(payload.inpaint?.growMaskBy ?? current.inpaint.growMaskBy, 0, 256)),
maskExpand: Math.round(clampNumber(payload.inpaint?.maskExpand ?? current.inpaint.maskExpand, -256, 256)),
maskFeather: Math.round(clampNumber(payload.inpaint?.maskFeather ?? current.inpaint.maskFeather, 0, 256)),
maskExpand: Math.round(clampNumber(payload.inpaint?.maskExpand ?? profileDefaults?.inpaint.maskExpand ?? current.inpaint.maskExpand, -256, 256)),
maskFeather: Math.round(clampNumber(payload.inpaint?.maskFeather ?? profileDefaults?.inpaint.maskFeather ?? current.inpaint.maskFeather, 0, 256)),
maskBlur: Math.round(clampNumber(payload.inpaint?.maskBlur ?? current.inpaint.maskBlur, 0, 256)),
maskDespeckle: Math.round(clampNumber(payload.inpaint?.maskDespeckle ?? current.inpaint.maskDespeckle, 0, 64)),
structureControl: payload.inpaint?.structureControl ?? profileDefaults?.inpaint.structureControl ?? current.inpaint.structureControl,
controlStrength: clampNumber(payload.inpaint?.controlStrength ?? profileDefaults?.inpaint.controlStrength ?? current.inpaint.controlStrength, 0, 1),
controlModel: payload.inpaint?.controlModel ?? current.inpaint.controlModel,
colorMatch: payload.inpaint?.colorMatch ?? profileDefaults?.inpaint.colorMatch ?? current.inpaint.colorMatch,
},
},
},
@@ -120,6 +145,26 @@ export const toolSetGenerateSettingsCommand: Command<ToolSetGenerateSettingsPayl
},
};
const modeByGenerateIntent: Record<GenerateIntent, GenerateSettings["mode"]> = {
create: "text-to-image",
replace: "inpaint",
remove: "inpaint",
extend: "outpaint",
variations: "image-to-image",
};
export const toolChooseGenerateIntentCommand: Command<ToolChooseGenerateIntentPayload> = {
id: commandIds.toolChooseGenerateIntent,
name: "Choose AI edit intent",
execute({ state }, payload) {
const mode = modeByGenerateIntent[payload.intent];
const architecture = generateArchitectureDefaults[state.editor.tools.generate.architecture].supportedModes.includes(mode)
? state.editor.tools.generate.architecture
: "sdxl";
return toolSetGenerateSettingsCommand.execute({ state }, { architecture, mode, inpaint: payload.intent === "remove" ? { profile: "remove" } : payload.intent === "replace" && state.editor.tools.generate.inpaint.profile === "remove" ? { profile: "replace" } : undefined });
},
};
export const toolSetBrushSettingsCommand: Command<ToolSetBrushSettingsPayload> = {
id: commandIds.toolSetBrushSettings,
name: "Set brush settings",
@@ -134,6 +179,33 @@ export const toolSetBrushSettingsCommand: Command<ToolSetBrushSettingsPayload> =
color: payload.color ?? state.editor.tools.brush.color,
size: clampNumber(payload.size ?? state.editor.tools.brush.size, 1, 200),
hardness: clampNumber(payload.hardness ?? state.editor.tools.brush.hardness, 0, 100),
opacity: clampNumber(payload.opacity ?? state.editor.tools.brush.opacity, 0, 100),
flow: clampNumber(payload.flow ?? state.editor.tools.brush.flow, 1, 100),
smoothing: clampNumber(payload.smoothing ?? state.editor.tools.brush.smoothing, 0, 100),
pressureSize: payload.pressureSize ?? state.editor.tools.brush.pressureSize,
},
},
},
};
},
};
export const toolSetFeatherSettingsCommand: Command<ToolSetFeatherSettingsPayload> = {
id: commandIds.toolSetFeatherSettings,
name: "Set feather settings",
execute({ state }, payload) {
return {
...state,
editor: {
...state.editor,
tools: {
...state.editor.tools,
feather: {
size: clampNumber(payload.size ?? state.editor.tools.feather.size, 1, 400),
radius: clampNumber(payload.radius ?? state.editor.tools.feather.radius, 1, 128),
strength: clampNumber(payload.strength ?? state.editor.tools.feather.strength, 1, 100),
smoothing: clampNumber(payload.smoothing ?? state.editor.tools.feather.smoothing, 0, 100),
pressureSize: payload.pressureSize ?? state.editor.tools.feather.pressureSize,
},
},
},
@@ -228,7 +300,13 @@ export const toolSetBrushStrokePreviewCommand: Command<ToolSetBrushStrokePreview
...state,
editor: {
...state.editor,
brushStrokePreview: { layerId: payload.layerId, assetId: payload.assetId, source: payload.source },
brushStrokePreview: {
layerId: payload.layerId,
assetId: payload.assetId,
source: payload.source,
pendingTargetLayerId: payload.pendingTargetLayerId,
intrinsicSize: payload.intrinsicSize,
},
},
};
},
@@ -250,6 +328,43 @@ export const toolSetMaskViewModeCommand: Command<ToolSetMaskViewModePayload> = {
},
};
export const toolBeginMaskShapeCommand: Command<ToolBeginMaskShapePayload> = {
id: commandIds.toolBeginMaskShape,
name: "Begin mask lasso",
execute({ state }, payload) {
const tool = state.editor.tools.activeTool;
if ((tool !== "maskLasso" && tool !== "maskRectangle") || state.editor.maskEdit?.kind !== "inpaintRegion") return state;
return { ...state, editor: { ...state.editor, maskShapeSession: { shape: tool === "maskRectangle" ? "rectangle" : "lasso", points: [{ ...payload.point }], mode: payload.mode } } };
},
};
export const toolAppendMaskShapeCommand: Command<ToolAppendMaskShapePayload> = {
id: commandIds.toolAppendMaskShape,
name: "Append mask lasso point",
history: { mode: "ignore" },
execute({ state }, payload) {
const session = state.editor.maskShapeSession;
if (!session) return state;
if (session.shape === "rectangle") {
const start = session.points[0];
if (!start) return state;
return { ...state, editor: { ...state.editor, maskShapeSession: { ...session, points: [start, { ...payload.point }] } } };
}
const previous = session.points[session.points.length - 1];
if (previous && Math.hypot(payload.point.x - previous.x, payload.point.y - previous.y) < 1) return state;
return { ...state, editor: { ...state.editor, maskShapeSession: { ...session, points: [...session.points, { ...payload.point }] } } };
},
};
export const toolClearMaskShapeCommand: Command = {
id: commandIds.toolClearMaskShape,
name: "Clear mask lasso",
history: { mode: "ignore" },
execute({ state }) {
return state.editor.maskShapeSession ? { ...state, editor: { ...state.editor, maskShapeSession: undefined } } : state;
},
};
export const toolEnterMaskEditCommand: Command<ToolEnterMaskEditPayload> = {
id: commandIds.toolEnterMaskEdit,
name: "Enter mask edit",
@@ -258,14 +373,41 @@ export const toolEnterMaskEditCommand: Command<ToolEnterMaskEditPayload> = {
const maskLocation = findLayerLocation(state.document, payload.maskLayerId);
if (!targetLocation || !maskLocation) return state;
if (getLayerMask(targetLocation.layer)?.maskLayerId !== payload.maskLayerId) return state;
if (maskLocation.layer.type === "group") return state;
if (maskLocation.layer.type !== "image" && maskLocation.layer.type !== "raster") return state;
return {
...state,
editor: {
...state.editor,
selection: { artboardId: targetLocation.artboardId, layerIds: [payload.targetLayerId] },
maskEdit: { targetLayerId: payload.targetLayerId, maskLayerId: payload.maskLayerId },
maskEdit: { kind: "layerMask", targetLayerId: payload.targetLayerId, maskLayerId: payload.maskLayerId, maskAssetId: maskLocation.layer.assetId },
brushPreview: undefined,
brushStrokePreview: undefined,
tools: {
...state.editor.tools,
activeTool: "brush",
interactionMode: { type: "tool", tool: "brush" },
},
},
};
},
};
export const toolEnterInpaintRegionEditCommand: Command<ToolEnterInpaintRegionEditPayload> = {
id: commandIds.toolEnterInpaintRegionEdit,
name: "Enter inpaint region edit",
execute({ state }, payload) {
const targetLocation = findLayerLocation(state.document, payload.targetLayerId);
const region = state.document.inpaintRegions.find((candidate) => candidate.id === payload.regionId && candidate.targetLayerId === payload.targetLayerId);
const maskAsset = region ? state.document.assets.find((asset) => asset.id === region.maskAssetId) : undefined;
if (!targetLocation || !region || !maskAsset) return state;
return {
...state,
editor: {
...state.editor,
selection: { artboardId: targetLocation.artboardId, layerIds: [payload.targetLayerId] },
maskEdit: { kind: "inpaintRegion", targetLayerId: payload.targetLayerId, inpaintRegionId: region.id, maskAssetId: region.maskAssetId, viewMode: "overlay" },
brushPreview: undefined,
brushStrokePreview: undefined,
tools: {
@@ -283,6 +425,7 @@ export const toolExitMaskEditCommand: Command = {
name: "Exit mask edit",
execute({ state }) {
if (!state.editor.maskEdit) return state;
const contextualTool = state.editor.tools.activeTool === "semanticSelect" || state.editor.tools.activeTool === "maskLasso" || state.editor.tools.activeTool === "maskRectangle";
return {
...state,
@@ -291,6 +434,8 @@ export const toolExitMaskEditCommand: Command = {
maskEdit: undefined,
brushPreview: undefined,
brushStrokePreview: undefined,
maskShapeSession: undefined,
tools: contextualTool ? { ...state.editor.tools, activeTool: "select", interactionMode: { type: "tool", tool: "select" } } : state.editor.tools,
},
};
},
@@ -339,13 +484,19 @@ export const toolExitTemporaryPanCommand: Command = {
export const toolCommands = [
toolSetActiveCommand,
toolSetGenerateSettingsCommand,
toolChooseGenerateIntentCommand,
toolSetBrushSettingsCommand,
toolSetFeatherSettingsCommand,
toolSetChromaKeySettingsCommand,
toolSetMagicWandSettingsCommand,
toolSetBrushPreviewCommand,
toolSetBrushStrokePreviewCommand,
toolSetMaskViewModeCommand,
toolBeginMaskShapeCommand,
toolAppendMaskShapeCommand,
toolClearMaskShapeCommand,
toolEnterMaskEditCommand,
toolEnterInpaintRegionEditCommand,
toolExitMaskEditCommand,
toolEnterTemporaryPanCommand,
toolExitTemporaryPanCommand,

View File

@@ -5,12 +5,13 @@ import type { Layer } from "@core/layer";
import { getLayerMask } from "@core/layer-mask-utils";
import { resolveTransformTargetBounds } from "@editor/transform-targets";
import type { TransformTarget } from "@editor/transform";
import { measureTextLayer } from "@core/text-layer";
export function applyTransformTargetBounds(document: ImageDocument, target: TransformTarget, bounds: Rect): ImageDocument {
if (target.type === "artboard") return { ...document, artboards: document.artboards.map((artboard) => artboard.id === target.id ? { ...artboard, bounds: { ...bounds } } : artboard) };
const layer = findLayer(document, target.id);
if (!layer) return document;
return layer.type === "group" ? applyGroupBounds(document, layer.id, bounds) : applyLeafBounds(document, layer.id, bounds);
return layer.type === "group" ? applyGroupBounds(document, layer.id, bounds) : layer.type === "adjustment" ? document : applyLeafBounds(document, layer.id, bounds);
}
function applyLeafBounds(document: ImageDocument, layerId: LayerId, bounds: Rect): ImageDocument {
@@ -39,15 +40,19 @@ function mapGroupBounds(document: ImageDocument, layers: Layer[], groupId: Layer
function scaleSubtree(document: ImageDocument, layer: Layer, initial: Rect, bounds: Rect, scale: { x: number; y: number }): Layer {
if (layer.type === "group") return { ...layer, children: layer.children.map((child) => scaleSubtree(document, child, initial, bounds, scale)) };
if (!document.assets.some((asset) => asset.id === layer.assetId)) return layer;
if (layer.type === "adjustment") return layer;
if (layer.type !== "text" && !document.assets.some((asset) => asset.id === layer.assetId)) return layer;
return { ...layer, transform: { ...layer.transform, position: { x: bounds.x + (layer.transform.position.x - initial.x) * scale.x, y: bounds.y + (layer.transform.position.y - initial.y) * scale.y }, scale: { x: layer.transform.scale.x * scale.x, y: layer.transform.scale.y * scale.y } } };
}
function mapLeafBounds(document: ImageDocument, layers: Layer[], layerId: LayerId, bounds: Rect): Layer[] {
return layers.map((layer) => {
if (layer.id === layerId && layer.type !== "group") {
const asset = document.assets.find((candidate) => candidate.id === layer.assetId);
return asset ? { ...layer, transform: { ...layer.transform, position: { x: bounds.x, y: bounds.y }, scale: { x: bounds.w / asset.intrinsicSize.w, y: bounds.h / asset.intrinsicSize.h } } } : layer;
if (layer.id === layerId && layer.type !== "group" && layer.type !== "adjustment") {
const asset = layer.type === "text" ? undefined : document.assets.find((candidate) => candidate.id === layer.assetId);
if (layer.type !== "text" && !asset) return layer;
const source = layer.type === "text" ? { x: 0, y: 0, ...measureTextLayer(layer) } : layer.sourceRect ?? { x: 0, y: 0, ...asset!.intrinsicSize };
const scale = { x: bounds.w / source.w, y: bounds.h / source.h };
return { ...layer, transform: { ...layer.transform, position: { x: bounds.x - source.x * scale.x, y: bounds.y - source.y * scale.y }, scale } };
}
return layer.type === "group" ? { ...layer, children: mapLeafBounds(document, layer.children, layerId, bounds) } : layer;
});

View File

@@ -1,7 +1,8 @@
import { describe, expect, test } from "bun:test";
import { createInitialAppState } from "@editor/initial-state";
import { documentAddArtboardCommand } from "./document";
import { transformBeginCommand, transformEndCommand, transformSetBoundsCommand, transformUpdateCommand } from "./transform";
import { transformBeginCommand, transformEndCommand, transformSetBoundsCommand, transformSetRotationCommand, transformUpdateCommand } from "./transform";
import { documentAddAssetCommand, documentAddRasterLayerCommand, documentAddLayerMaskCommand, documentSetLayerLockedCommand } from "./document";
function artboardState() {
return documentAddArtboardCommand.execute(
@@ -79,6 +80,19 @@ describe("transform commands", () => {
expect(updated.document.artboards[0]?.bounds).toEqual({ x: 4, y: 19, w: 1, h: 1 });
});
test("sets leaf rotation and keeps its attached mask aligned", () => {
let state = artboardState();
state = documentAddAssetCommand.execute({ state }, { asset: { id: "asset", name: "Asset", mimeType: "image/png", source: "asset", intrinsicSize: { w: 10, h: 10 } } });
state = documentAddRasterLayerCommand.execute({ state }, { artboardId: "a1", layer: { id: "layer", type: "raster", name: "Layer", visible: true, locked: false, opacity: 1, assetId: "asset", transform: { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 } } });
state = documentAddLayerMaskCommand.execute({ state }, { layerId: "layer", asset: { id: "mask-asset", name: "Mask", mimeType: "image/png", source: "mask", intrinsicSize: { w: 10, h: 10 } }, maskLayer: { id: "mask", type: "raster", name: "Mask", visible: true, locked: false, opacity: 1, assetId: "mask-asset", transform: { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 } } });
const rotated = transformSetRotationCommand.execute({ state }, { target: { type: "layer", id: "layer" }, rotation: Math.PI / 2 });
expect(rotated.document.artboards[0]?.layers.map((layer) => layer.transform.rotation)).toEqual([Math.PI / 2, Math.PI / 2]);
const locked = documentSetLayerLockedCommand.execute({ state: rotated }, { layerId: "layer", locked: true });
expect(transformSetRotationCommand.execute({ state: locked }, { target: { type: "layer", id: "layer" }, rotation: 0 })).toBe(locked);
});
test("ends transform session", () => {
const started = transformBeginCommand.execute(
{ state: artboardState() },

View File

@@ -1,8 +1,10 @@
import type { Rect, Vec2D } from "@core/geometry";
import type { Layer } from "@core/layer";
import { applyTransformTargetBounds } from "./transform-document";
import type { TransformHandle, TransformTarget } from "@editor/transform";
import type { Command } from "./command";
import { commandIds } from "./ids";
import type { AppState } from "@editor/state";
export type TransformBeginPayload = {
target: TransformTarget;
@@ -21,6 +23,11 @@ export type TransformSetBoundsPayload = {
bounds: Rect;
};
export type TransformSetRotationPayload = {
target: TransformTarget;
rotation: number;
};
export const transformBeginCommand: Command<TransformBeginPayload> = {
id: commandIds.transformBegin,
name: "Begin transform",
@@ -70,6 +77,7 @@ export const transformSetBoundsCommand: Command<TransformSetBoundsPayload> = {
id: commandIds.transformSetBounds,
name: "Set transform bounds",
execute({ state }, payload) {
if (isTargetLocked(state, payload.target)) return state;
return {
...state,
document: applyTransformTargetBounds(state.document, payload.target, normalizeRect(payload.bounds)),
@@ -77,6 +85,42 @@ export const transformSetBoundsCommand: Command<TransformSetBoundsPayload> = {
},
};
export const transformSetRotationCommand: Command<TransformSetRotationPayload> = {
id: commandIds.transformSetRotation,
name: "Set transform rotation",
execute({ state }, payload) {
if (payload.target.type !== "layer" || !Number.isFinite(payload.rotation) || isTargetLocked(state, payload.target)) return state;
const location = state.document.artboards.flatMap((artboard) => findLayerInTree(artboard.layers, payload.target.id)).find(Boolean);
if (!location || location.type === "group") return state;
const maskId = "layerMask" in location ? location.layerMask?.maskLayerId : undefined;
const ids = new Set([payload.target.id, ...(maskId ? [maskId] : [])]);
return {
...state,
document: {
...state.document,
artboards: state.document.artboards.map((artboard) => ({ ...artboard, layers: mapRotation(artboard.layers, ids, payload.rotation) })),
},
};
},
};
function isTargetLocked(state: AppState, target: TransformTarget) {
if (target.type === "artboard") return state.document.artboards.find((artboard) => artboard.id === target.id)?.locked !== false;
return state.document.artboards.flatMap((artboard) => findLayerInTree(artboard.layers, target.id)).find(Boolean)?.locked !== false;
}
function findLayerInTree(layers: Layer[], id: string): Layer[] {
return layers.flatMap((layer) => layer.id === id ? [layer] : layer.type === "group" ? findLayerInTree(layer.children, id) : []);
}
function mapRotation(layers: Layer[], ids: ReadonlySet<string>, rotation: number): Layer[] {
return layers.map((layer) => ({
...layer,
...(ids.has(layer.id) ? { transform: { ...layer.transform, rotation } } : {}),
...(layer.type === "group" ? { children: mapRotation(layer.children, ids, rotation) } : {}),
})) as Layer[];
}
export const transformEndCommand: Command = {
id: commandIds.transformEnd,
name: "End transform",
@@ -94,7 +138,7 @@ export const transformEndCommand: Command = {
},
};
export const transformCommands = [transformBeginCommand, transformUpdateCommand, transformSetBoundsCommand, transformEndCommand] satisfies Command<unknown>[];
export const transformCommands = [transformBeginCommand, transformUpdateCommand, transformSetBoundsCommand, transformSetRotationCommand, transformEndCommand] satisfies Command<unknown>[];
function transformBounds(bounds: Rect, handle: TransformHandle, delta: Vec2D, constrained = false): Rect {
if (handle === "body") {

View File

@@ -0,0 +1,23 @@
import { describe, expect, test } from "bun:test";
import { createInitialAppState } from "@editor/initial-state";
import { workspaceSetPanelCommand } from "./workspace";
describe("workspace commands", () => {
test.each(["generate", "chromaKey"] as const)("opens the %s operation without changing the persistent tool", (panel) => {
const initial = createInitialAppState("Test");
const state = {
...initial,
editor: {
...initial.editor,
tools: { ...initial.editor.tools, activeTool: "brush" as const, interactionMode: { type: "tool" as const, tool: "brush" as const } },
brushPreview: { position: { x: 10, y: 20 } },
brushStrokePreview: { layerId: "layer", assetId: "asset", source: "preview" },
},
};
const next = workspaceSetPanelCommand.execute({ state }, { panel });
expect(next.editor.workspace.panel).toBe(panel);
expect(next.editor.tools.activeTool).toBe("brush");
expect(next.editor.brushPreview).toBeUndefined();
expect(next.editor.brushStrokePreview).toBeUndefined();
});
});

View File

@@ -10,23 +10,19 @@ export const workspaceSetPanelCommand: Command<WorkspaceSetPanelPayload> = {
history: { mode: "ignore" },
execute({ state }, payload) {
if (!workspacePanels.has(payload.panel)) return state;
const currentTool = state.editor.tools.activeTool;
const previousNonGenerateTool = currentTool === "generate" ? state.editor.workspace.previousNonGenerateTool : currentTool;
const nextTool = payload.panel === "generate" ? "generate" : currentTool === "generate" ? previousNonGenerateTool : currentTool;
if (state.editor.workspace.panel === payload.panel && currentTool === nextTool) return state;
if (state.editor.workspace.panel === payload.panel && !state.editor.brushPreview && !state.editor.brushStrokePreview) return state;
return {
...state,
editor: {
...state.editor,
workspace: { panel: payload.panel, previousNonGenerateTool },
tools: nextTool === currentTool ? state.editor.tools : { ...state.editor.tools, activeTool: nextTool, interactionMode: { type: "tool", tool: nextTool } },
brushPreview: nextTool === currentTool ? state.editor.brushPreview : undefined,
brushStrokePreview: nextTool === currentTool ? state.editor.brushStrokePreview : undefined,
workspace: { panel: payload.panel },
brushPreview: undefined,
brushStrokePreview: undefined,
},
};
},
};
const workspacePanels = new Set<WorkspacePanel>(["none", "generate", "layers"]);
const workspacePanels = new Set<WorkspacePanel>(["none", "generate", "chromaKey", "layers"]);
export const workspaceCommands = [workspaceSetPanelCommand] satisfies Command<unknown>[];

26
core/adjustment-layer.ts Normal file
View File

@@ -0,0 +1,26 @@
import type { BaseLayer } from "./base-layer";
export type ColorAdjustment = {
brightness: number;
contrast: number;
saturation: number;
colorBalance: { red: number; green: number; blue: number };
};
/**
* A non-destructive adjustment affecting pixels already composited beneath it
* on its artboard. Adjustment layers are top-level artboard nodes so live WebGL
* and exported PNG compositing have identical scope. Transform and masks are
* intentionally unsupported.
*/
export type AdjustmentLayer = BaseLayer & {
type: "adjustment";
adjustment: ColorAdjustment;
};
export const neutralColorAdjustment: ColorAdjustment = {
brightness: 0,
contrast: 0,
saturation: 0,
colorBalance: { red: 0, green: 0, blue: 0 },
};

View File

@@ -1,5 +1,5 @@
import type { Rect, Size } from "./geometry";
import type { AssetId, GenerationCandidateId, LayerId } from "./id";
import type { AssetId, GenerationCandidateId, InpaintRegionId, LayerId } from "./id";
export type GeneratedAssetMode = "text-to-image" | "image-to-image" | "inpaint" | "outpaint";
@@ -26,10 +26,13 @@ export type AssetGenerationProvenance = {
scheduler: string;
width: number;
height: number;
batchSize: number;
refinePass: boolean;
refineStrength: number;
};
inpaint?: {
targetLayerId: LayerId;
maskLayerId: LayerId;
regionId: InpaintRegionId;
sourceAssetId: AssetId;
maskAssetId: AssetId;
crop: {
@@ -49,6 +52,11 @@ export type AssetGenerationProvenance = {
maskFeather: number;
maskExpand: number;
cropPadding: number;
profile: string;
structureControl: string;
controlStrength: number;
controlModel: string;
colorMatch: boolean;
};
};
};

View File

@@ -1,6 +1,7 @@
import type { Artboard } from "./artboard";
import type { Asset } from "./asset";
import type { DocumentId } from "./id";
import type { InpaintRegion } from "./inpaint-region";
export type ImageDocument = {
id: DocumentId;
@@ -8,4 +9,5 @@ export type ImageDocument = {
version: number;
artboards: Artboard[];
assets: Asset[];
inpaintRegions: InpaintRegion[];
};

View File

@@ -5,5 +5,6 @@ export type DocumentId = OpaqueId<"DocumentId">;
export type ArtboardId = OpaqueId<"ArtboardId">;
export type LayerId = OpaqueId<"LayerId">;
export type AssetId = OpaqueId<"AssetId">;
export type InpaintRegionId = OpaqueId<"InpaintRegionId">;
export type GenerationCandidateId = OpaqueId<"GenerationCandidateId">;
export type GenerationJobId = OpaqueId<"GenerationJobId">;

View File

@@ -1,7 +1,10 @@
import type { BaseLayer } from "./base-layer";
import type { AssetId } from "./id";
import type { Rect } from "./geometry";
export type ImageLayer = BaseLayer & {
type: "image";
assetId: AssetId;
/** Non-destructive source pixels displayed by this layer. Absence means the full asset. */
sourceRect?: Rect;
};

View File

@@ -13,10 +13,15 @@ export type {
Transform,
Vec2D,
} from "./geometry";
export type { ArtboardId, AssetId, DocumentId, GenerationCandidateId, GenerationJobId, LayerId } from "./id";
export type { ArtboardId, AssetId, DocumentId, GenerationCandidateId, GenerationJobId, InpaintRegionId, LayerId } from "./id";
export type { InpaintRegion } from "./inpaint-region";
export type { ImageLayer } from "./image-layer";
export type { AdjustmentLayer, ColorAdjustment } from "./adjustment-layer";
export { neutralColorAdjustment } from "./adjustment-layer";
export type { Layer } from "./layer";
export type { LayerMask } from "./layer-mask";
export { getLayerMask, hasLayerMask } from "./layer-mask-utils";
export type { LayerGroup } from "./layer-group";
export type { RasterLayer } from "./raster-layer";
export type { TextAlignment, TextFontFamily, TextFontStyle, TextFontWeight, TextLayer, TextStyle } from "./text-layer";
export { builtInTextFonts, isValidTextStyle, measureTextLayer } from "./text-layer";

9
core/inpaint-region.ts Normal file
View File

@@ -0,0 +1,9 @@
import type { AssetId, InpaintRegionId, LayerId } from "./id";
export type InpaintRegion = {
id: InpaintRegionId;
name: string;
targetLayerId: LayerId;
maskAssetId: AssetId;
enabled: boolean;
};

View File

@@ -1,5 +1,7 @@
import type { ImageLayer } from "./image-layer";
import type { LayerGroup } from "./layer-group";
import type { RasterLayer } from "./raster-layer";
import type { AdjustmentLayer } from "./adjustment-layer";
import type { TextLayer } from "./text-layer";
export type Layer = ImageLayer | RasterLayer | LayerGroup;
export type Layer = ImageLayer | RasterLayer | LayerGroup | AdjustmentLayer | TextLayer;

View File

@@ -1,7 +1,10 @@
import type { BaseLayer } from "./base-layer";
import type { AssetId } from "./id";
import type { Rect } from "./geometry";
export type RasterLayer = BaseLayer & {
type: "raster";
assetId: AssetId;
/** Non-destructive source pixels displayed by this layer. Absence means the full asset. */
sourceRect?: Rect;
};

13
core/text-layer.test.ts Normal file
View File

@@ -0,0 +1,13 @@
import { describe, expect, test } from "bun:test";
import { isValidTextStyle, measureTextLayer, type TextLayer } from "./text-layer";
const layer: TextLayer = { id: "text-1", type: "text", name: "Title", visible: true, locked: false, opacity: 1, transform: { position: { x: 10, y: 20 }, scale: { x: 1, y: 1 }, rotation: 0 }, content: "Hello\nWorld", style: { fontFamily: "Arial", fontSize: 20, fontWeight: 400, fontStyle: "normal", color: "#ffffff", alignment: "left", lineHeight: 1.25 } };
describe("text layer metrics", () => {
test("uses deterministic multiline bounds", () => expect(measureTextLayer(layer)).toEqual({ w: 54, h: 50 }));
test("validates the closed built-in typography policy", () => {
expect(isValidTextStyle(layer.style)).toBeTrue();
expect(isValidTextStyle({ ...layer.style, fontFamily: "Remote Font" as never })).toBeFalse();
expect(isValidTextStyle({ ...layer.style, color: "red" })).toBeFalse();
});
});

44
core/text-layer.ts Normal file
View File

@@ -0,0 +1,44 @@
import type { BaseLayer } from "./base-layer";
import type { Size } from "./geometry";
export const builtInTextFonts = ["Arial", "Georgia", "Courier New", "Trebuchet MS"] as const;
export type TextFontFamily = typeof builtInTextFonts[number];
export type TextAlignment = "left" | "center" | "right";
export type TextFontStyle = "normal" | "italic";
export type TextFontWeight = 400 | 700;
export type TextStyle = {
fontFamily: TextFontFamily;
fontSize: number;
fontWeight: TextFontWeight;
fontStyle: TextFontStyle;
color: string;
alignment: TextAlignment;
lineHeight: number;
};
export type TextLayer = BaseLayer & {
type: "text";
content: string;
style: TextStyle;
};
/** Deterministic layout box shared by selection, renderer and export. */
export function measureTextLayer(layer: Pick<TextLayer, "content" | "style">): Size {
const lines = layer.content.split("\n");
const weightFactor = layer.style.fontWeight === 700 ? 1.04 : 1;
const italicFactor = layer.style.fontStyle === "italic" ? 1.03 : 1;
const familyFactor = layer.style.fontFamily === "Courier New" ? 0.62 : layer.style.fontFamily === "Georgia" ? 0.56 : 0.54;
const longest = Math.max(1, ...lines.map((line) => [...line].length));
return {
w: Math.max(1, longest * layer.style.fontSize * familyFactor * weightFactor * italicFactor),
h: Math.max(1, lines.length * layer.style.fontSize * layer.style.lineHeight),
};
}
export function isValidTextStyle(style: TextStyle): boolean {
return builtInTextFonts.includes(style.fontFamily) && Number.isFinite(style.fontSize) && style.fontSize >= 1 && style.fontSize <= 1000
&& (style.fontWeight === 400 || style.fontWeight === 700) && (style.fontStyle === "normal" || style.fontStyle === "italic")
&& /^#[0-9a-f]{6}$/i.test(style.color) && (style.alignment === "left" || style.alignment === "center" || style.alignment === "right")
&& Number.isFinite(style.lineHeight) && style.lineHeight >= 0.5 && style.lineHeight <= 5;
}

View File

@@ -4,6 +4,7 @@ import type { Layer } from "@core/layer";
import { createDocumentReadIndex, forEachLayerBackToFront, resolveIndexedLayerBounds } from "./document-indexes";
const document: ImageDocument = {
inpaintRegions: [],
id: "d1",
name: "Indexed Document",
version: 1,
@@ -58,6 +59,11 @@ describe("document read indexes", () => {
expect(resolveIndexedLayerBounds(index, raster("missing", "Missing", "missing-asset"))).toBeUndefined();
});
test("resolves a cropped layer from its retained source-pixel position", () => {
const cropped = { ...document, artboards: [{ ...document.artboards[0]!, layers: [{ ...raster("cropped", "Cropped", "asset-target", { x: 10, y: 20 }, { x: 2, y: 3 }), sourceRect: { x: 5, y: 4, w: 20, h: 10 } }] }] };
expect(resolveIndexedLayerBounds(createDocumentReadIndex(cropped), "cropped")).toEqual({ x: 20, y: 32, w: 40, h: 30 });
});
test("visits layers back to front without mutating source order", () => {
const layers = document.artboards[0]!.layers;
const visited: string[] = [];
@@ -67,6 +73,17 @@ describe("document read indexes", () => {
expect(visited).toEqual(["group", "target", "mask"]);
expect(layers.map((layer) => layer.id)).toEqual(["mask", "target", "group"]);
});
test("indexes pathological nesting without recursive stack overflow", () => {
let layers: Layer[] = [raster("leaf", "Leaf", "asset-nested", { x: 80, y: 10 }, { x: 2, y: 3 })];
for (let depth = 0; depth < 10_000; depth += 1) layers = [group(`group-${depth}`, `Group ${depth}`, layers)];
const deepDocument = { ...document, artboards: [{ ...document.artboards[0]!, layers }] };
const index = createDocumentReadIndex(deepDocument);
expect(index.layerById.size).toBe(10_001);
expect(resolveIndexedLayerBounds(index, layers[0]!)).toEqual({ x: 80, y: 10, w: 40, h: 30 });
});
});
function raster(

View File

@@ -4,6 +4,7 @@ import type { Rect } from "@core/geometry";
import type { ArtboardId, AssetId, LayerId } from "@core/id";
import type { Layer } from "@core/layer";
import { getLayerMask } from "@core/layer-mask-utils";
import { measureTextLayer } from "@core/text-layer";
export type IndexedLayerInfo = {
artboardId: ArtboardId;
@@ -75,16 +76,23 @@ export function resolveIndexedLayerBounds(index: DocumentReadIndex, layerOrId: L
switch (layer.type) {
case "group":
return unionLayerBounds(index, layer.children);
case "adjustment":
return undefined;
case "text": {
const size = measureTextLayer(layer);
return { x: layer.transform.position.x, y: layer.transform.position.y, w: size.w * layer.transform.scale.x, h: size.h * layer.transform.scale.y };
}
case "image":
case "raster": {
const asset = index.assetById.get(layer.assetId);
if (!asset) return undefined;
const source = layer.sourceRect ?? { x: 0, y: 0, ...asset.intrinsicSize };
return {
x: layer.transform.position.x,
y: layer.transform.position.y,
w: asset.intrinsicSize.w * layer.transform.scale.x,
h: asset.intrinsicSize.h * layer.transform.scale.y,
x: layer.transform.position.x + source.x * layer.transform.scale.x,
y: layer.transform.position.y + source.y * layer.transform.scale.y,
w: source.w * layer.transform.scale.x,
h: source.h * layer.transform.scale.y,
};
}
}
@@ -99,55 +107,59 @@ function indexLayerTree(options: {
documentMaskLayerIds: Set<LayerId>;
maskLayerIdsByLayerList: Map<readonly Layer[], ReadonlySet<LayerId>>;
}): Set<LayerId> {
const layerListMaskLayerIds = new Set<LayerId>();
for (let index = 0; index < options.layers.length; index += 1) {
const layer = options.layers[index];
type Frame = { layers: readonly Layer[]; parentGroupId?: LayerId; visited: boolean };
const stack: Frame[] = [{ layers: options.layers, parentGroupId: options.parentGroupId, visited: false }];
while (stack.length > 0) {
const frame = stack.pop();
if (!frame) continue;
if (frame.visited) {
const ids = new Set<LayerId>();
for (const layer of frame.layers) {
const mask = getLayerMask(layer);
if (mask) ids.add(mask.maskLayerId);
if (layer.type === "group") for (const id of options.maskLayerIdsByLayerList.get(layer.children) ?? []) ids.add(id);
}
options.maskLayerIdsByLayerList.set(frame.layers, ids);
continue;
}
stack.push({ ...frame, visited: true });
for (let index = frame.layers.length - 1; index >= 0; index -= 1) {
const layer = frame.layers[index];
if (!layer) continue;
options.layerById.set(layer.id, layer);
options.layerInfoById.set(layer.id, {
artboardId: options.artboardId,
parentGroupId: options.parentGroupId,
layer,
siblings: options.layers,
index,
});
const layerMask = getLayerMask(layer);
if (layerMask) {
options.documentMaskLayerIds.add(layerMask.maskLayerId);
layerListMaskLayerIds.add(layerMask.maskLayerId);
}
if (layer.type === "group") {
const childMaskLayerIds = indexLayerTree({
...options,
layers: layer.children,
parentGroupId: layer.id,
});
for (const maskLayerId of childMaskLayerIds) layerListMaskLayerIds.add(maskLayerId);
options.layerInfoById.set(layer.id, { artboardId: options.artboardId, parentGroupId: frame.parentGroupId, layer, siblings: frame.layers, index });
const mask = getLayerMask(layer);
if (mask) options.documentMaskLayerIds.add(mask.maskLayerId);
if (layer.type === "group") stack.push({ layers: layer.children, parentGroupId: layer.id, visited: false });
}
}
options.maskLayerIdsByLayerList.set(options.layers, layerListMaskLayerIds);
return layerListMaskLayerIds;
return new Set(options.maskLayerIdsByLayerList.get(options.layers) ?? []);
}
function countDisplayLayers(layers: readonly Layer[], maskLayerIds: ReadonlySet<LayerId>): number {
let count = 0;
for (const layer of layers) {
const stack = [...layers];
while (stack.length > 0) {
const layer = stack.pop();
if (!layer) continue;
if (maskLayerIds.has(layer.id)) continue;
count += 1;
if (layer.type === "group") count += countDisplayLayers(layer.children, maskLayerIds);
if (layer.type === "group") stack.push(...layer.children);
}
return count;
}
function unionLayerBounds(index: DocumentReadIndex, layers: readonly Layer[]): Rect | undefined {
let bounds: Rect | undefined;
for (const layer of layers) {
const stack = [...layers];
while (stack.length > 0) {
const layer = stack.pop();
if (!layer) continue;
if (layer.type === "group") {
stack.push(...layer.children);
continue;
}
if (layer.type === "adjustment") continue;
const layerBounds = resolveIndexedLayerBounds(index, layer);
if (!layerBounds) continue;
bounds = bounds ? unionRects(bounds, layerBounds) : layerBounds;

View File

@@ -1,5 +1,5 @@
export type { AppState, EditorState, SelectionState, ViewportState } from "./state";
export type { BrushSettings, ChromaKeySettings, MagicWandSettings, InteractionMode, ToolId, ToolState } from "./tools";
export type { BrushSettings, ChromaKeySettings, FeatherSettings, MagicWandSettings, InteractionMode, OperationId, ToolId, ToolState } from "./tools";
export { initialToolState } from "./tools";
export { createInitialAppState, initialEditorState } from "./initial-state";
export type { AppStore, StateListener } from "./store";

View File

@@ -26,12 +26,12 @@ export const initialEditorState: EditorState = {
},
workspace: {
panel: "none",
previousNonGenerateTool: "select",
},
transformSession: undefined,
maskEdit: undefined,
brushPreview: undefined,
brushStrokePreview: undefined,
maskShapeSession: undefined,
pointerSession: undefined,
};
@@ -43,6 +43,7 @@ export function createInitialAppState(name = "Untitled"): AppState {
version: 1,
artboards: [],
assets: [],
inpaintRegions: [],
},
editor: initialEditorState,
history: { past: [], future: [] },

View File

@@ -1,7 +1,7 @@
import type { ImageDocument } from "@core/document";
import type { Angle, Rect, Size, Transform, Vec2D } from "@core/geometry";
import type { ArtboardId, AssetId, GenerationCandidateId, GenerationJobId, LayerId } from "@core/id";
import type { GenerateArchitecture, GenerateMode, GenerateSettings, ToolId, ToolState } from "./tools";
import type { ArtboardId, AssetId, GenerationCandidateId, GenerationJobId, InpaintRegionId, LayerId } from "@core/id";
import type { GenerateArchitecture, GenerateMode, GenerateSettings, ToolState } from "./tools";
import type { TransformSession } from "./transform";
export type ViewportState = {
@@ -19,8 +19,11 @@ export type SelectionState = {
export type MaskViewMode = "composite" | "blackWhite" | "alpha" | "overlay";
export type MaskEditState = {
kind: "layerMask" | "inpaintRegion";
targetLayerId: LayerId;
maskLayerId: LayerId;
maskAssetId: AssetId;
maskLayerId?: LayerId;
inpaintRegionId?: InpaintRegionId;
viewMode?: MaskViewMode;
};
@@ -32,6 +35,14 @@ export type BrushStrokePreviewState = {
layerId: LayerId;
assetId: AssetId;
source: string;
pendingTargetLayerId?: LayerId;
intrinsicSize?: Size;
};
export type MaskShapeSession = {
shape: "lasso" | "rectangle";
points: Vec2D[];
mode: "replace" | "add" | "subtract";
};
export type GenerationCandidate = {
@@ -42,10 +53,12 @@ export type GenerationCandidate = {
mode: GenerateSettings["mode"];
settings: GenerateSettings;
seed: number;
favorite?: boolean;
width: number;
height: number;
inputImage?: string;
maskImage?: string;
blendMaskImage?: string;
placement: {
artboardId: ArtboardId;
layerName: string;
@@ -53,11 +66,14 @@ export type GenerationCandidate = {
};
inpaint?: {
targetLayerId: LayerId;
maskLayerId: LayerId;
regionId: InpaintRegionId;
sourceAssetId: AssetId;
maskAssetId: AssetId;
inputImage: string;
maskImage: string;
editMaskImage: string;
blendMaskImage: string;
revision: { source: string; mask: string };
crop: {
assetBounds: Rect;
documentBounds: Rect;
@@ -81,8 +97,8 @@ export type GenerationCandidate = {
export type GenerationCompareMode = "result" | "before" | "split";
export type GenerationJobKind = "generate" | "regenerate" | "refine" | "replace";
export type GenerationJobStatus = "running" | "succeeded" | "failed";
export type GenerationJobKind = "generate" | "regenerate" | "refine" | "replace" | "mask";
export type GenerationJobStatus = "running" | "succeeded" | "failed" | "cancelled";
export type GenerationJob = {
id: GenerationJobId;
@@ -92,6 +108,8 @@ export type GenerationJob = {
startedAt: number;
finishedAt?: number;
error?: string;
progress?: number;
detail?: string;
};
export type GenerationState = {
@@ -113,10 +131,15 @@ export type GenerationArchitectureOption = {
export type GenerationOptions = {
architectures?: GenerationArchitectureOption[];
models?: string[];
inpaintModels?: string[];
textEncoders?: string[];
vaes?: string[];
samplers?: string[];
schedulers?: string[];
controlModels?: string[];
structureControls?: Array<"canny" | "depth" | "pose">;
semanticSelection?: boolean;
sam3Models?: string[];
};
export type GenerationResourcesState = { status: "idle" | "loading" | "ready" | "failed"; options?: GenerationOptions; error?: string };
@@ -127,11 +150,14 @@ export type CommandPaletteState = {
selectedIndex: number;
};
export type WorkspacePanel = "none" | "generate" | "layers";
export type WorkspacePanel = "none" | "generate" | "chromaKey" | "layers";
export function isOperationWorkspacePanel(panel: WorkspacePanel): boolean {
return panel === "generate" || panel === "chromaKey";
}
export type WorkspaceState = {
panel: WorkspacePanel;
previousNonGenerateTool: ToolId;
};
export type EditorState = {
@@ -145,6 +171,7 @@ export type EditorState = {
maskEdit?: MaskEditState;
brushPreview?: BrushPreviewState;
brushStrokePreview?: BrushStrokePreviewState;
maskShapeSession?: MaskShapeSession;
pointerSession?: { type: "pan" };
};

View File

@@ -1,6 +1,9 @@
export const availableToolIds = ["select", "generate", "brush", "eraser", "chromaKey", "magicWand", "pan"] as const;
export const availableToolIds = ["select", "brush", "eraser", "feather", "magicWand", "semanticSelect", "maskLasso", "maskRectangle", "pan"] as const;
export const availableOperationIds = ["generate", "chromaKey"] as const;
export type ToolId = (typeof availableToolIds)[number];
export type OperationId = (typeof availableOperationIds)[number];
export type InteractionMode =
| { type: "tool"; tool: ToolId }
@@ -10,6 +13,18 @@ export type BrushSettings = {
color: string;
size: number;
hardness: number;
opacity: number;
flow: number;
smoothing: number;
pressureSize: boolean;
};
export type FeatherSettings = {
size: number;
radius: number;
strength: number;
smoothing: number;
pressureSize: boolean;
};
export type ChromaKeySettings = {
@@ -25,12 +40,15 @@ export type ChromaKeySettings = {
export type MagicWandMode = "replace" | "add" | "subtract";
export type GenerateMode = "text-to-image" | "image-to-image" | "inpaint" | "outpaint";
export type GenerateIntent = "create" | "replace" | "remove" | "extend" | "variations";
export const generateArchitectures = ["sdxl", "z-image", "z-image-turbo", "anima"] as const;
export type GenerateArchitecture = (typeof generateArchitectures)[number];
export type GenerateModel = string;
export type InpaintMaskedContent = "neutral" | "original" | "originalColor" | "edges";
export type InpaintProfile = "remove" | "replace" | "repair" | "material" | "reshape" | "custom";
export type InpaintStructureControl = "none" | "canny" | "depth" | "pose";
export type GenerateSettings = {
architecture: GenerateArchitecture;
@@ -48,6 +66,9 @@ export type GenerateSettings = {
scheduler: string;
width: number;
height: number;
batchSize: number;
refinePass: boolean;
refineStrength: number;
outpaint: {
left: number;
top: number;
@@ -56,6 +77,7 @@ export type GenerateSettings = {
feathering: number;
};
inpaint: {
profile: InpaintProfile;
maskedAreaOnly: boolean;
cropPadding: number;
maskPolarity: "hidden" | "revealed";
@@ -65,6 +87,10 @@ export type GenerateSettings = {
maskFeather: number;
maskBlur: number;
maskDespeckle: number;
structureControl: InpaintStructureControl;
controlStrength: number;
controlModel: string;
colorMatch: boolean;
};
};
@@ -97,6 +123,7 @@ export type ToolState = {
activeTool: ToolId;
interactionMode: InteractionMode;
brush: BrushSettings;
feather: FeatherSettings;
chromaKey: ChromaKeySettings;
magicWand: MagicWandSettings;
generate: GenerateSettings;
@@ -105,7 +132,8 @@ export type ToolState = {
export const initialToolState: ToolState = {
activeTool: "select",
interactionMode: { type: "tool", tool: "select" },
brush: { color: "#111827", size: 8, hardness: 100 },
brush: { color: "#111827", size: 8, hardness: 100, opacity: 100, flow: 100, smoothing: 20, pressureSize: true },
feather: { size: 96, radius: 16, strength: 65, smoothing: 25, pressureSize: true },
chromaKey: { color: "#00ff00", tolerance: 32, softness: 24, feather: 0, choke: 0, despeckle: 0, spill: 50 },
magicWand: { tolerance: 32, feather: 0, choke: 0, despeckle: 0, contiguous: true, mode: "replace" },
generate: {
@@ -124,11 +152,22 @@ export const initialToolState: ToolState = {
scheduler: "normal",
width: 1024,
height: 1024,
batchSize: 4,
refinePass: true,
refineStrength: 20,
outpaint: { left: 128, top: 128, right: 128, bottom: 128, feathering: 32 },
inpaint: { maskedAreaOnly: true, cropPadding: 96, maskPolarity: "hidden", maskedContent: "neutral", growMaskBy: 6, maskExpand: 0, maskFeather: 0, maskBlur: 0, maskDespeckle: 0 },
inpaint: { profile: "replace", maskedAreaOnly: true, cropPadding: 128, maskPolarity: "revealed", maskedContent: "neutral", growMaskBy: 6, maskExpand: 8, maskFeather: 3, maskBlur: 0, maskDespeckle: 0, structureControl: "none", controlStrength: 0.55, controlModel: "auto", colorMatch: true },
},
};
export const inpaintProfileDefaults: Record<Exclude<InpaintProfile, "custom">, Omit<Partial<GenerateSettings>, "inpaint"> & { inpaint: Partial<GenerateSettings["inpaint"]> }> = {
remove: { strength: 100, inpaint: { maskedContent: "neutral", cropPadding: 160, maskExpand: 12, maskFeather: 4, structureControl: "none" } },
replace: { strength: 85, inpaint: { maskedContent: "neutral", cropPadding: 128, maskExpand: 8, maskFeather: 3, structureControl: "none" } },
repair: { strength: 45, refinePass: true, refineStrength: 15, inpaint: { maskedContent: "original", cropPadding: 96, maskExpand: 4, maskFeather: 2, structureControl: "none" } },
material: { strength: 65, refinePass: true, refineStrength: 20, inpaint: { maskedContent: "edges", cropPadding: 128, maskExpand: 6, maskFeather: 3, structureControl: "canny", controlStrength: 0.55, colorMatch: false } },
reshape: { strength: 90, inpaint: { maskedContent: "neutral", cropPadding: 160, maskExpand: 10, maskFeather: 4, structureControl: "depth", controlStrength: 0.4 } },
};
export function isPanInteractionMode(interactionMode: InteractionMode): boolean {
return interactionMode.type === "temporary-pan" || (interactionMode.type === "tool" && interactionMode.tool === "pan");
}

View File

@@ -4,6 +4,7 @@ import { resolveTransformTargetBounds, selectedTransformTarget } from "./transfo
import { applyTransformTargetBounds } from "@commands/transform-document";
const document: ImageDocument = {
inpaintRegions: [],
id: "d1",
name: "Test",
version: 1,
@@ -56,6 +57,13 @@ describe("transform targets", () => {
expect(layer?.transform).toEqual({ position: { x: 30, y: 40 }, scale: { x: 2, y: 0.5 }, rotation: 0 });
});
test("applies visible bounds to a cropped layer while retaining its source crop", () => {
const cropped = { ...document, artboards: [{ ...document.artboards[0]!, layers: [{ ...document.artboards[0]!.layers[0]!, sourceRect: { x: 10, y: 5, w: 40, h: 20 } }] }] };
const next = applyTransformTargetBounds(cropped, { type: "layer", id: "l1" }, { x: 100, y: 80, w: 200, h: 60 });
expect(next.artboards[0]?.layers[0]).toMatchObject({ sourceRect: { x: 10, y: 5, w: 40, h: 20 }, transform: { position: { x: 50, y: 65 }, scale: { x: 5, y: 3 } } });
expect(resolveTransformTargetBounds(next, { type: "layer", id: "l1" })).toEqual({ x: 100, y: 80, w: 200, h: 60 });
});
test("keeps attached mask bounds in sync with transformed layers", () => {
const maskedDocument: ImageDocument = {
...document,

View File

@@ -3,6 +3,7 @@ import type { Rect } from "@core/geometry";
import type { Layer } from "@core/layer";
import type { ArtboardId, LayerId } from "@core/id";
import type { TransformTarget } from "./transform";
import { measureTextLayer } from "@core/text-layer";
export function resolveTransformTargetBounds(document: ImageDocument, target: TransformTarget): Rect | undefined {
switch (target.type) {
@@ -53,14 +54,21 @@ function resolveLayerBounds(document: ImageDocument, layer: Layer): Rect | undef
case "raster": {
const asset = document.assets.find((candidate) => candidate.id === layer.assetId);
if (!asset) return undefined;
const source = layer.sourceRect ?? { x: 0, y: 0, ...asset.intrinsicSize };
return {
x: layer.transform.position.x,
y: layer.transform.position.y,
w: asset.intrinsicSize.w * layer.transform.scale.x,
h: asset.intrinsicSize.h * layer.transform.scale.y,
x: layer.transform.position.x + source.x * layer.transform.scale.x,
y: layer.transform.position.y + source.y * layer.transform.scale.y,
w: source.w * layer.transform.scale.x,
h: source.h * layer.transform.scale.y,
};
}
case "text": {
const size = measureTextLayer(layer);
return { x: layer.transform.position.x, y: layer.transform.position.y, w: size.w * layer.transform.scale.x, h: size.h * layer.transform.scale.y };
}
case "adjustment":
return undefined;
}
}

View File

@@ -6,6 +6,7 @@ const server = serve({
routes: {
"/api/comfy/models": handleComfyApi,
"/api/comfy/generate": handleComfyApi,
"/api/comfy/segment": handleComfyApi,
"/*": index,
},

View File

@@ -17,7 +17,7 @@ describe("command palette input", () => {
expect(dispatched).toEqual([{ commandId: commandIds.commandPaletteOpen, payload: undefined }]);
});
test("ignores unmodified K so the chroma key tool keeps its shortcut", () => {
test("ignores unmodified K so the chroma key operation keeps its shortcut", () => {
const consumed = handleCommandPaletteKey({
event: { key: "k", code: "KeyK", altKey: false, ctrlKey: false, metaKey: false, shiftKey: false },
dispatch: () => undefined as never,

View File

@@ -1,5 +1,6 @@
import type { Rect, Size, Vec2D } from "@core/geometry";
import type { InputArtboardId, InputDocument, InputLayer, InputLayerId } from "./read-model";
import { measureTextLayer } from "@core/text-layer";
export type InputViewportState = {
center: Vec2D;
@@ -90,6 +91,12 @@ function resolveLayerBounds(document: InputDocument, layer: InputLayer): Rect |
h: asset.intrinsicSize.h * layer.transform.scale.y,
};
}
case "text": {
const size = measureTextLayer(layer);
return { x: layer.transform.position.x, y: layer.transform.position.y, w: size.w * layer.transform.scale.x, h: size.h * layer.transform.scale.y };
}
case "adjustment":
return undefined;
}
}

View File

@@ -23,6 +23,7 @@ export function pointerInputEventFromPointerEvent(event: PointerEvent): PointerI
pointerType: normalizePointerType(event.pointerType),
position: { x: event.offsetX, y: event.offsetY },
buttons: event.buttons,
pressure: event.pointerType === "pen" ? event.pressure : 1,
altKey: event.altKey,
ctrlKey: event.ctrlKey,
metaKey: event.metaKey,

View File

@@ -2,11 +2,20 @@ import { commandIds } from "@commands/ids";
import type { Dispatch } from "@commands/dispatcher";
import type { KeybindEvent } from "./keyboard";
export function handleHistoryKey(options: { event: KeybindEvent; dispatch: Dispatch }): boolean {
export type HistoryActions = { undo(): void; redo(): void };
export function handleHistoryKey(options: { event: KeybindEvent; dispatch?: Dispatch; actions?: HistoryActions }): boolean {
if (options.event.altKey) return false;
const modifier = options.event.metaKey || options.event.ctrlKey;
if (!modifier || options.event.key.toLowerCase() !== "z") return false;
if (options.actions) {
if (options.event.shiftKey) options.actions.redo();
else options.actions.undo();
} else if (options.dispatch) {
options.dispatch(options.event.shiftKey ? commandIds.historyRedo : commandIds.historyUndo, undefined);
} else {
return false;
}
return true;
}

View File

@@ -8,6 +8,7 @@ export { handleKeybind, keybindFromEvent } from "./keyboard";
export { handleCommandPaletteKey } from "./command-palette";
export { handleHistoryKey } from "./history";
export { handleToolKey } from "./tool-keybinds";
export { handleOperationKey } from "./operation-keybinds";
export { findGroup, findLayerInfoInDocument, handleDeleteSelectionKey, resolveLayerDrop } from "./layers-panel";
export type { LayerDropTarget, LayerInfo } from "./layers-panel";
export { handleArtboardSelection } from "./selection";

View File

@@ -0,0 +1,22 @@
import { describe, expect, test } from "bun:test";
import { commandIds } from "@commands/ids";
import type { Dispatch } from "@commands/dispatcher";
import { handleOperationKey } from "./operation-keybinds";
describe("operation keybinds", () => {
test.each([
["g", "generate"],
["k", "chromaKey"],
] as const)("opens the %s operation without selecting a tool", (key, panel) => {
const dispatched: unknown[] = [];
const consumed = handleOperationKey({
event: { key, code: `Key${key.toUpperCase()}`, altKey: false, ctrlKey: false, metaKey: false, shiftKey: false },
dispatch: ((commandId: unknown, payload: unknown) => {
dispatched.push({ commandId, payload });
return undefined;
}) as unknown as Dispatch,
});
expect(consumed).toBe(true);
expect(dispatched).toEqual([{ commandId: commandIds.workspaceSetPanel, payload: { panel } }]);
});
});

View File

@@ -0,0 +1,16 @@
import { commandIds } from "@commands/ids";
import type { Dispatch } from "@commands/dispatcher";
import type { KeybindEvent } from "./keyboard";
const operationKeybinds = {
g: "generate",
k: "chromaKey",
} as const;
export function handleOperationKey(options: { event: KeybindEvent; dispatch: Dispatch }): boolean {
if (options.event.altKey || options.event.ctrlKey || options.event.metaKey) return false;
const panel = operationKeybinds[options.event.key.toLowerCase() as keyof typeof operationKeybinds];
if (!panel) return false;
options.dispatch(commandIds.workspaceSetPanel, { panel });
return true;
}

View File

@@ -5,6 +5,7 @@ export type PointerInputEvent = {
pointerType: "mouse" | "pen" | "touch";
position: Vec2D;
buttons: number;
pressure?: number;
altKey: boolean;
ctrlKey: boolean;
metaKey: boolean;

View File

@@ -1,4 +1,5 @@
import type { Rect, Size, Transform } from "@core/geometry";
import type { TextStyle } from "@core/text-layer";
export type InputLayerId = string;
export type InputArtboardId = string;
@@ -16,6 +17,8 @@ type InputBaseLayer = {
export type InputLayer =
| (InputBaseLayer & { type: "group"; children: InputLayer[] })
| (InputBaseLayer & { type: "adjustment" })
| (InputBaseLayer & { type: "text"; content: string; style: TextStyle })
| (InputBaseLayer & { type: "image" | "raster"; assetId: string });
export type InputDocument = {

View File

@@ -0,0 +1,20 @@
import { describe, expect, test } from "bun:test";
import { commandIds } from "@commands/ids";
import type { Dispatch } from "@commands/dispatcher";
import { handleToolKey } from "./tool-keybinds";
describe("tool keybinds", () => {
test("selects the feather tool with F", () => {
const dispatched: unknown[] = [];
const consumed = handleToolKey({
event: { key: "f", code: "KeyF", altKey: false, ctrlKey: false, metaKey: false, shiftKey: false },
dispatch: ((commandId: unknown, payload: unknown) => {
dispatched.push({ commandId, payload });
return undefined;
}) as unknown as Dispatch,
});
expect(consumed).toBe(true);
expect(dispatched).toEqual([{ commandId: commandIds.toolSetActive, payload: { tool: "feather" } }]);
});
});

View File

@@ -3,10 +3,9 @@ import type { Dispatch } from "@commands/dispatcher";
import type { KeybindEvent } from "./keyboard";
const toolKeybinds = {
g: "generate",
b: "brush",
e: "eraser",
k: "chromaKey",
f: "feather",
w: "magicWand",
p: "pan",
s: "select",

View File

@@ -16,7 +16,7 @@ import type { PointerInputEvent } from "./pointer";
type TransformHandle = "body" | "nw" | "n" | "ne" | "e" | "se" | "s" | "sw" | "w";
type InputToolId = "select" | "generate" | "brush" | "eraser" | "chromaKey" | "magicWand" | "pan";
type InputToolId = "select" | "brush" | "eraser" | "feather" | "magicWand" | "semanticSelect" | "maskLasso" | "maskRectangle" | "pan";
type InputInteractionMode =
| { type: "tool"; tool: InputToolId }

View File

@@ -7,6 +7,7 @@ import type { DocumentReadIndex, IndexedLayerInfo } from "@editor/document-index
import { resolveIndexedLayerBounds } from "@editor/document-indexes";
import type { SelectionState } from "@editor/state";
import type { AppStore } from "@editor/store";
import { neutralColorAdjustment } from "@core/adjustment-layer";
export function addArtboard(document: ImageDocument, dispatch: AppStore["dispatch"]) {
const index = document.artboards.length + 1;
@@ -62,6 +63,20 @@ export function addGroupLayer(artboardId: ArtboardId, dispatch: AppStore["dispat
dispatch(commandIds.documentAddGroupLayer, { artboardId, group: createGroup("Group") });
}
export function addAdjustmentLayer(artboardId: ArtboardId, dispatch: AppStore["dispatch"]) {
dispatch(commandIds.documentAddAdjustmentLayer, { artboardId, layer: { id: crypto.randomUUID(), type: "adjustment", name: "Color adjustment", visible: true, locked: false, opacity: 1, transform: { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 }, adjustment: { ...neutralColorAdjustment, colorBalance: { ...neutralColorAdjustment.colorBalance } } } });
}
export function addTextLayer(document: ImageDocument, artboardId: ArtboardId, selectedLayer: IndexedLayerInfo | undefined, dispatch: AppStore["dispatch"]) {
const artboard = document.artboards.find((candidate) => candidate.id === artboardId);
if (!artboard) return;
dispatch(commandIds.documentAddTextLayer, { artboardId, parentGroupId: selectedLayer?.layer.type === "group" ? selectedLayer.layer.id : undefined, layer: {
id: crypto.randomUUID(), type: "text", name: "Text", visible: true, locked: false, opacity: 1,
transform: { position: { x: artboard.bounds.x + 40, y: artboard.bounds.y + 40 }, scale: { x: 1, y: 1 }, rotation: 0 },
content: "Text", style: { fontFamily: "Arial", fontSize: 48, fontWeight: 400, fontStyle: "normal", color: "#ffffff", alignment: "left", lineHeight: 1.2 },
} });
}
export function groupLayers(artboardId: ArtboardId, layerIds: string[], dispatch: AppStore["dispatch"]) {
dispatch(commandIds.documentGroupLayers, { artboardId, layerIds, group: createGroup("Group") });
}
@@ -105,7 +120,7 @@ export function moveLayer(documentIndex: DocumentReadIndex, info: IndexedLayerIn
export function addLayerMask(documentIndex: DocumentReadIndex, layerInfo: IndexedLayerInfo, dispatch: AppStore["dispatch"]) {
const layer = layerInfo.layer;
if (layer.type === "group") return;
if (layer.type === "group" || layer.type === "adjustment" || layer.type === "text") return;
const asset = documentIndex.assetById.get(layer.assetId);
const bounds = resolveIndexedLayerBounds(documentIndex, layer);
@@ -143,6 +158,25 @@ export function addLayerMask(documentIndex: DocumentReadIndex, layerInfo: Indexe
});
}
export function duplicateLayer(documentIndex: DocumentReadIndex, layerInfo: IndexedLayerInfo, dispatch: AppStore["dispatch"]) {
const ids = collectDuplicateIds(layerInfo.layer, layerInfo.siblings);
dispatch(commandIds.documentDuplicateLayer, {
layerId: layerInfo.layer.id,
idByLayerId: Object.fromEntries(ids.map((id) => [id, crypto.randomUUID()])),
});
}
function collectDuplicateIds(layer: Layer, siblings: readonly Layer[]): string[] {
const ids = collectTreeIds(layer);
const maskId = getLayerMask(layer)?.maskLayerId;
const mask = maskId ? siblings.find((candidate) => candidate.id === maskId) : undefined;
return mask ? [...collectTreeIds(mask), ...ids] : ids;
}
function collectTreeIds(layer: Layer): string[] {
return [layer.id, ...(layer.type === "group" ? layer.children.flatMap(collectTreeIds) : [])];
}
const emptyLayerIds = new Set<string>();
function createGroup(name: string): Extract<Layer, { type: "group" }> {

View File

@@ -1,16 +1,24 @@
import type { ImageDocument } from "@core/document";
import type { GenerationCandidate } from "@editor/state";
import { loadImageCanvas, maskValueFromRgba } from "@platform/browser/maskRaster";
import { createContentRevision } from "./inpaintPrep";
export async function createMaskedPixelReplacementSource(document: ImageDocument, candidate: GenerationCandidate): Promise<string> {
if (!candidate.inpaint) throw new Error("Only inpaint candidates can replace masked pixels.");
const targetAsset = document.assets.find((asset) => asset.id === candidate.inpaint?.sourceAssetId);
if (!targetAsset) throw new Error("The source layer for this candidate no longer exists.");
const region = document.inpaintRegions.find((item) => item.id === candidate.inpaint?.regionId);
const maskAsset = region ? document.assets.find((asset) => asset.id === region.maskAssetId) : undefined;
if (!region || !maskAsset) throw new Error("The AI edit region for this candidate no longer exists.");
const [sourceRevision, maskRevision] = await Promise.all([createContentRevision(targetAsset.source), createContentRevision(maskAsset.source)]);
if (sourceRevision !== candidate.inpaint.revision.source || maskRevision !== candidate.inpaint.revision.mask) {
throw new Error("The source or AI edit region changed after generation. Rebuild candidates from the current edit region before replacing pixels.");
}
const targetCanvas = await loadImageCanvas(targetAsset.source, targetAsset.intrinsicSize.w, targetAsset.intrinsicSize.h);
const generatedCanvas = await loadImageCanvas(candidate.source, candidate.width, candidate.height);
const maskCanvas = await loadImageCanvas(candidate.inpaint.maskImage, candidate.width, candidate.height);
const maskCanvas = await loadImageCanvas(candidate.inpaint.blendMaskImage, candidate.width, candidate.height);
const targetContext = require2dContext(targetCanvas);
const generatedContext = require2dContext(generatedCanvas);
@@ -19,6 +27,7 @@ export async function createMaskedPixelReplacementSource(document: ImageDocument
const generatedData = generatedContext.getImageData(0, 0, generatedCanvas.width, generatedCanvas.height);
const maskData = maskContext.getImageData(0, 0, maskCanvas.width, maskCanvas.height);
const crop = candidate.inpaint.crop.assetBounds;
const colorOffset = candidate.settings.inpaint.colorMatch ? boundaryColorOffset(targetData.data, generatedData.data, maskData.data, candidate.width, candidate.height, crop, targetCanvas.width, targetCanvas.height) : [0, 0, 0];
for (let y = 0; y < candidate.height; y += 1) {
for (let x = 0; x < candidate.width; x += 1) {
@@ -33,7 +42,8 @@ export async function createMaskedPixelReplacementSource(document: ImageDocument
for (let channel = 0; channel < 4; channel += 1) {
const previous = targetData.data[targetIndex + channel] ?? 0;
const next = generatedData.data[generatedIndex + channel] ?? previous;
const rawNext = generatedData.data[generatedIndex + channel] ?? previous;
const next = channel < 3 ? Math.max(0, Math.min(255, rawNext + (colorOffset[channel] ?? 0))) : rawNext;
targetData.data[targetIndex + channel] = Math.round(previous * (1 - mask) + next * mask);
}
}
@@ -43,6 +53,28 @@ export async function createMaskedPixelReplacementSource(document: ImageDocument
return targetCanvas.toDataURL("image/png");
}
function boundaryColorOffset(target: Uint8ClampedArray, generated: Uint8ClampedArray, mask: Uint8ClampedArray, width: number, height: number, crop: { x: number; y: number }, targetWidth: number, targetHeight: number): number[] {
const targetTotal = [0, 0, 0];
const generatedTotal = [0, 0, 0];
let count = 0;
for (let y = 0; y < height; y += 1) for (let x = 0; x < width; x += 1) {
const index = (y * width + x) * 4;
const amount = maskValueFromRgba(mask, index) / 255;
if (amount <= 0.05 || amount >= 0.65) continue;
const targetX = Math.round(crop.x) + x;
const targetY = Math.round(crop.y) + y;
if (targetX < 0 || targetY < 0 || targetX >= targetWidth || targetY >= targetHeight) continue;
const targetIndex = (targetY * targetWidth + targetX) * 4;
for (let channel = 0; channel < 3; channel += 1) {
targetTotal[channel] = (targetTotal[channel] ?? 0) + (target[targetIndex + channel] ?? 0);
generatedTotal[channel] = (generatedTotal[channel] ?? 0) + (generated[index + channel] ?? 0);
}
count += 1;
}
if (count < 16) return [0, 0, 0];
return targetTotal.map((total, channel) => Math.max(-32, Math.min(32, total / count - (generatedTotal[channel] ?? 0) / count)));
}
function require2dContext(canvas: HTMLCanvasElement): CanvasRenderingContext2D {
const context = canvas.getContext("2d");
if (!context) throw new Error("Unable to prepare generated candidate");

View File

@@ -6,7 +6,8 @@ export async function runGenerationJob(options: {
kind: GenerationJobKind;
label: string;
dispatch: AppStore["dispatch"];
task: () => Promise<void>;
signal: AbortSignal;
task: (signal: AbortSignal, report: (progress: number, detail: string) => void) => Promise<void>;
}): Promise<void> {
const jobId = crypto.randomUUID();
const startedAt = Date.now();
@@ -14,9 +15,13 @@ export async function runGenerationJob(options: {
if (!nextState.editor.generation.jobs.some((job) => job.id === jobId && job.status === "running")) return;
try {
await options.task();
await options.task(options.signal, (progress, detail) => options.dispatch(commandIds.generationUpdateJob, { jobId, progress, detail }));
options.dispatch(commandIds.generationSucceedJob, { jobId, finishedAt: Date.now() });
} catch (reason: unknown) {
if (options.signal.aborted) {
options.dispatch(commandIds.generationCancelJob, { jobId, finishedAt: Date.now() });
return;
}
options.dispatch(commandIds.generationFailJob, {
jobId,
finishedAt: Date.now(),

View File

@@ -1,22 +1,28 @@
import type { Asset } from "@core/asset";
import type { ImageDocument } from "@core/document";
import type { Rect } from "@core/geometry";
import type { AssetId, InpaintRegionId, LayerId } from "@core/id";
import type { Layer } from "@core/layer";
import { getLayerMask } from "@core/layer-mask-utils";
import type { SelectionState } from "@editor/state";
import type { GenerateSettings } from "@editor/tools";
import { createDocumentReadIndex, resolveIndexedLayerBounds } from "@editor/document-indexes";
import { createNormalizedMaskSource, cropCanvas, cropMaskValuesToDataUrl, expandRectWithinBounds, loadImageCanvas } from "@platform/browser/maskRaster";
import { createNormalizedMaskSource, cropCanvas, cropMaskValuesToDataUrl, expandRectWithinBounds, loadImageCanvas, sampleDocumentCanvasInLayerSpace } from "@platform/browser/maskRaster";
import { renderArtboardCanvas } from "@platform/browser/exportArtboardPng";
export type InpaintBundle = {
inputImage: string;
sourceImage: string;
contextImage: string;
maskImage: string;
editMaskImage: string;
blendMaskImage: string;
width: number;
height: number;
targetLayerId: string;
maskLayerId: string;
sourceAssetId: string;
maskAssetId: string;
targetLayerId: LayerId;
regionId: InpaintRegionId;
sourceAssetId: AssetId;
maskAssetId: AssetId;
revision: { source: string; mask: string };
crop: {
assetBounds: Rect;
documentBounds: Rect;
@@ -46,10 +52,8 @@ type InpaintTarget = {
artboardId: string;
layer: Extract<Layer, { type: "image" | "raster" }>;
asset: Asset;
bounds: Rect;
maskLayer: Extract<Layer, { type: "image" | "raster" }>;
maskAsset: Asset;
maskBounds: Rect;
regionId: InpaintRegionId;
};
const modelMultiple = 8;
@@ -62,41 +66,65 @@ export async function buildInpaintBundle(document: ImageDocument, selection: Sel
const width = Math.max(1, Math.round(target.asset.intrinsicSize.w));
const height = Math.max(1, Math.round(target.asset.intrinsicSize.h));
const normalizedMask = await createNormalizedMaskSource(target.maskAsset.source, width, height, {
polarity: settings.inpaint.maskPolarity,
const sourceLimit = target.layer.sourceRect ?? { x: 0, y: 0, w: width, h: height };
const editMask = await createNormalizedMaskSource(target.maskAsset.source, width, height, {
polarity: "revealed",
despeckle: settings.inpaint.maskDespeckle,
limit: sourceLimit,
});
const noiseMask = await createNormalizedMaskSource(target.maskAsset.source, width, height, {
polarity: "revealed",
expand: settings.inpaint.maskExpand,
feather: settings.inpaint.maskFeather,
blur: settings.inpaint.maskBlur,
despeckle: settings.inpaint.maskDespeckle,
limit: sourceLimit,
});
const blendMask = await createNormalizedMaskSource(target.maskAsset.source, width, height, {
polarity: "revealed",
feather: settings.inpaint.maskFeather,
despeckle: settings.inpaint.maskDespeckle,
limit: sourceLimit,
});
if (!normalizedMask.bounds) throw new Error("The selected layer mask has no inpaint pixels.");
if (!editMask.bounds || !noiseMask.bounds) throw new Error("Paint over the area you want AI to replace.");
const crop = settings.inpaint.maskedAreaOnly
? expandRectWithinBounds(normalizedMask.bounds, settings.inpaint.cropPadding, { w: width, h: height }, modelMultiple, minModelSize)
? expandRectWithinBounds(noiseMask.bounds, settings.inpaint.cropPadding, { w: width, h: height }, modelMultiple, minModelSize)
: { x: 0, y: 0, w: width, h: height };
const outputWidth = toModelSize(crop.w, "width");
const outputHeight = toModelSize(crop.h, "height");
const inputCanvas = prepareMaskedContentInputCanvas(await loadImageCanvas(target.asset.source, width, height), normalizedMask.values, settings.inpaint.maskedContent);
const inputImage = cropCanvas(inputCanvas, crop, outputWidth, outputHeight);
const maskImage = cropMaskValuesToDataUrl(normalizedMask.values, width, height, crop, outputWidth, outputHeight);
const scaleX = target.bounds.w / width;
const scaleY = target.bounds.h / height;
const sourceCanvas = await loadImageCanvas(target.asset.source, width, height);
const sourceImage = cropCanvas(sourceCanvas, crop, outputWidth, outputHeight);
const preparedCanvas = prepareMaskedContentInputCanvas(await loadImageCanvas(target.asset.source, width, height), editMask.values, settings.inpaint.maskedContent);
const preparedSourceImage = cropCanvas(preparedCanvas, crop, outputWidth, outputHeight);
const artboard = document.artboards.find((candidate) => candidate.id === target.artboardId);
if (!artboard) throw new Error("The target artboard no longer exists.");
const contextImage = sampleDocumentCanvasInLayerSpace(await renderArtboardCanvas(artboard, document.assets), artboard.bounds, target.layer, target.asset.intrinsicSize, crop, outputWidth, outputHeight);
const inputImage = await mergePreparedRegionIntoContext(contextImage, preparedSourceImage, cropMaskValuesToDataUrl(editMask.values, width, height, crop, outputWidth, outputHeight), outputWidth, outputHeight);
const maskImage = cropMaskValuesToDataUrl(noiseMask.values, width, height, crop, outputWidth, outputHeight);
const editMaskImage = cropMaskValuesToDataUrl(editMask.values, width, height, crop, outputWidth, outputHeight);
const blendMaskImage = cropMaskValuesToDataUrl(blendMask.values, width, height, crop, outputWidth, outputHeight);
const scaleX = target.layer.transform.scale.x;
const scaleY = target.layer.transform.scale.y;
const documentBounds = {
x: target.bounds.x + crop.x * scaleX,
y: target.bounds.y + crop.y * scaleY,
x: target.layer.transform.position.x + crop.x * scaleX,
y: target.layer.transform.position.y + crop.y * scaleY,
w: outputWidth * scaleX,
h: outputHeight * scaleY,
};
return {
inputImage,
sourceImage,
contextImage,
maskImage,
editMaskImage,
blendMaskImage,
width: outputWidth,
height: outputHeight,
targetLayerId: target.layer.id,
maskLayerId: target.maskLayer.id,
regionId: target.regionId,
sourceAssetId: target.asset.id,
maskAssetId: target.maskAsset.id,
crop: {
@@ -106,14 +134,14 @@ export async function buildInpaintBundle(document: ImageDocument, selection: Sel
maskedAreaOnly: settings.inpaint.maskedAreaOnly,
},
mask: {
polarity: settings.inpaint.maskPolarity,
activeBounds: normalizedMask.bounds,
polarity: "revealed",
activeBounds: editMask.bounds,
},
placement: {
artboardId: target.artboardId,
layerName: `${target.layer.name} inpaint`,
transform: {
position: { x: documentBounds.x, y: documentBounds.y },
position: rotatedCropPosition(target.layer, target.asset, { ...crop, w: outputWidth, h: outputHeight }),
scale: { x: documentBounds.w / outputWidth, y: documentBounds.h / outputHeight },
rotation: target.layer.transform.rotation,
},
@@ -126,6 +154,10 @@ export async function buildInpaintBundle(document: ImageDocument, selection: Sel
maskExpand: settings.inpaint.maskExpand,
cropPadding: settings.inpaint.cropPadding,
},
revision: {
source: await createContentRevision(target.asset.source),
mask: await createContentRevision(target.maskAsset.source),
},
};
}
@@ -134,24 +166,18 @@ function resolveInpaintTarget(document: ImageDocument, selection: SelectionState
const documentIndex = createDocumentReadIndex(document);
const layerInfo = documentIndex.layerInfoById.get(selection.layerIds[0]);
if (!layerInfo || layerInfo.layer.type === "group") throw new Error("Select one image or raster layer to inpaint.");
if (!layerInfo || (layerInfo.layer.type !== "image" && layerInfo.layer.type !== "raster")) throw new Error("Select one image or raster layer to inpaint. Text must be rasterized first.");
const asset = documentIndex.assetById.get(layerInfo.layer.assetId);
if (!asset) throw new Error("The selected layer is missing its source image.");
const layerMask = getLayerMask(layerInfo.layer);
if (!layerMask?.enabled) throw new Error("Add a layer mask before running inpaint.");
const region = document.inpaintRegions.find((candidate) => candidate.targetLayerId === layerInfo.layer.id && candidate.enabled);
if (!region) throw new Error("Add an AI edit region before running inpaint.");
const maskAsset = documentIndex.assetById.get(region.maskAssetId);
if (!maskAsset) throw new Error("The AI edit region is missing its mask data.");
const maskLayer = documentIndex.layerById.get(layerMask.maskLayerId);
if (!maskLayer || maskLayer.type === "group") throw new Error("The selected layer mask is missing.");
if (!resolveIndexedLayerBounds(documentIndex, layerInfo.layer)) throw new Error("Unable to resolve the selected layer bounds.");
const maskAsset = documentIndex.assetById.get(maskLayer.assetId);
if (!maskAsset) throw new Error("The selected layer mask is missing its image data.");
const bounds = resolveIndexedLayerBounds(documentIndex, layerInfo.layer);
const maskBounds = resolveIndexedLayerBounds(documentIndex, maskLayer);
if (!bounds || !maskBounds) throw new Error("Unable to resolve the selected layer and mask bounds.");
return { artboardId: layerInfo.artboardId, layer: layerInfo.layer, asset, bounds, maskLayer, maskAsset, maskBounds };
return { artboardId: layerInfo.artboardId, layer: layerInfo.layer, asset, maskAsset, regionId: region.id };
}
function validateInpaintTarget(target: InpaintTarget) {
@@ -160,13 +186,6 @@ function validateInpaintTarget(target: InpaintTarget) {
if (Math.round(target.asset.intrinsicSize.w) !== Math.round(target.maskAsset.intrinsicSize.w) || Math.round(target.asset.intrinsicSize.h) !== Math.round(target.maskAsset.intrinsicSize.h)) {
throw new Error("The selected layer and mask image sizes do not match.");
}
if (!rectsAligned(target.bounds, target.maskBounds) || Math.abs(target.layer.transform.rotation - target.maskLayer.transform.rotation) > 0.001) {
throw new Error("The selected layer and mask are not aligned.");
}
}
function rectsAligned(a: Rect, b: Rect): boolean {
return Math.abs(a.x - b.x) <= 0.5 && Math.abs(a.y - b.y) <= 0.5 && Math.abs(a.w - b.w) <= 0.5 && Math.abs(a.h - b.h) <= 0.5;
}
function toModelSize(value: number, axis: "width" | "height"): number {
@@ -258,3 +277,52 @@ function blendChannel(previous: number, next: number, amount: number): number {
function clampNumber(value: number, min: number, max: number): number {
return Math.max(min, Math.min(max, value));
}
async function mergePreparedRegionIntoContext(contextSource: string, preparedSource: string, editMaskSource: string, width: number, height: number): Promise<string> {
const [contextCanvas, preparedCanvas, maskCanvas] = await Promise.all([
loadImageCanvas(contextSource, width, height),
loadImageCanvas(preparedSource, width, height),
loadImageCanvas(editMaskSource, width, height),
]);
const context = contextCanvas.getContext("2d");
const prepared = preparedCanvas.getContext("2d");
const mask = maskCanvas.getContext("2d");
if (!context || !prepared || !mask) return contextSource;
const contextData = context.getImageData(0, 0, width, height);
const preparedData = prepared.getImageData(0, 0, width, height);
const maskData = mask.getImageData(0, 0, width, height);
for (let pixel = 0; pixel < width * height; pixel += 1) {
const amount = (maskData.data[pixel * 4] ?? 0) / 255;
if (amount <= 0) continue;
for (let channel = 0; channel < 4; channel += 1) {
const index = pixel * 4 + channel;
contextData.data[index] = blendChannel(contextData.data[index] ?? 0, preparedData.data[index] ?? 0, amount);
}
}
context.putImageData(contextData, 0, 0);
return contextCanvas.toDataURL("image/png");
}
function rotatedCropPosition(layer: InpaintTarget["layer"], asset: Asset, crop: Rect) {
const source = layer.sourceRect ?? { x: 0, y: 0, ...asset.intrinsicSize };
const scale = layer.transform.scale;
const originalCenter = {
x: layer.transform.position.x + (source.x + source.w / 2) * scale.x,
y: layer.transform.position.y + (source.y + source.h / 2) * scale.y,
};
const cropCenter = {
x: layer.transform.position.x + (crop.x + crop.w / 2) * scale.x,
y: layer.transform.position.y + (crop.y + crop.h / 2) * scale.y,
};
const dx = cropCenter.x - originalCenter.x;
const dy = cropCenter.y - originalCenter.y;
const cos = Math.cos(layer.transform.rotation);
const sin = Math.sin(layer.transform.rotation);
const rotatedCenter = { x: originalCenter.x + dx * cos - dy * sin, y: originalCenter.y + dx * sin + dy * cos };
return { x: rotatedCenter.x - crop.w * scale.x / 2, y: rotatedCenter.y - crop.h * scale.y / 2 };
}
export async function createContentRevision(source: string): Promise<string> {
const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(source));
return Array.from(new Uint8Array(digest), (byte) => byte.toString(16).padStart(2, "0")).join("");
}

View File

@@ -0,0 +1,23 @@
import { describe, expect, test } from "bun:test";
import { initialToolState } from "@editor/tools";
import { resolveGenerationModeOptions, resolveGenerationModelOptions } from "./options";
describe("generation compatibility options", () => {
test("uses backend compatibility data for modes", () => {
const settings = { ...initialToolState.generate, architecture: "anima" as const, mode: "inpaint" as const };
const modes = [
{ value: "text-to-image" as const, label: "Text" },
{ value: "inpaint" as const, label: "Inpaint" },
];
const options = { architectures: [{ value: "anima" as const, label: "Anima", models: [], defaultModel: "auto", supportedModes: ["text-to-image" as const] }], models: [], samplers: [], schedulers: [], textEncoders: [], vaes: [] };
expect(resolveGenerationModeOptions(settings, options, modes).map((option) => option.value)).toEqual(["inpaint", "text-to-image"]);
});
test("keeps the current model alongside discovered models", () => {
const settings = { ...initialToolState.generate, model: "current.safetensors" };
const options = { architectures: [], models: ["found.safetensors"], samplers: [], schedulers: [], textEncoders: [], vaes: [] };
expect(resolveGenerationModelOptions(settings, options).map((option) => option.value)).toEqual(["auto", "found.safetensors", "current.safetensors"]);
});
});

View File

@@ -0,0 +1,41 @@
import type { GenerationOptions } from "@editor/state";
import { generateArchitectureDefaults } from "@editor/tools";
import type { GenerateMode, GenerateModel, GenerateSettings } from "@editor/tools";
export type GenerationSelectOption<TValue extends string> = { value: TValue; label: string };
export function resolveGenerationModelOptions(settings: GenerateSettings, options: GenerationOptions | undefined): readonly GenerationSelectOption<GenerateModel>[] {
const architecture = options?.architectures?.find((item) => item.value === settings.architecture);
const models = architecture?.models ?? (settings.architecture === "sdxl" ? options?.models : undefined) ?? [];
const fallbackModel = architecture?.defaultModel ?? generateArchitectureDefaults[settings.architecture].model;
const values = unique(["auto", ...models, ...(models.length === 0 && fallbackModel !== "auto" ? [fallbackModel] : []), settings.model]);
return values.map((model) => ({ value: model, label: model === "auto" ? "Auto" : model }));
}
export function resolveGenerationSupportOptions(settings: GenerateSettings, options: GenerationOptions | undefined) {
const defaults = generateArchitectureDefaults[settings.architecture];
return {
textEncoders: resolveGenerationStringOptions([...(options?.textEncoders ?? []), defaults.textEncoder].filter((value) => value !== "auto"), settings.textEncoder),
vaes: resolveGenerationStringOptions([...(options?.vaes ?? []), defaults.vae].filter((value) => value !== "auto"), settings.vae),
};
}
export function resolveGenerationStringOptions(values: string[] | undefined, current: string): readonly GenerationSelectOption<string>[] {
return unique([...(values ?? []), current]).map((value) => ({ value, label: value }));
}
export function resolveGenerationModeOptions(
settings: GenerateSettings,
options: GenerationOptions | undefined,
modes: readonly GenerationSelectOption<GenerateMode>[],
): readonly GenerationSelectOption<GenerateMode>[] {
const architecture = options?.architectures?.find((item) => item.value === settings.architecture);
const supportedModes = architecture?.supportedModes?.length ? architecture.supportedModes : generateArchitectureDefaults[settings.architecture].supportedModes;
const availableModes = modes.filter((mode) => supportedModes.includes(mode.value));
if (availableModes.some((mode) => mode.value === settings.mode)) return availableModes;
return [modes.find((mode) => mode.value === settings.mode), ...availableModes].filter((mode): mode is GenerationSelectOption<GenerateMode> => Boolean(mode));
}
function unique<T>(values: T[]): T[] {
return Array.from(new Set(values));
}

View File

@@ -0,0 +1,94 @@
import { describe, expect, test } from "bun:test";
import type { ImageDocument } from "@core/document";
import type { GenerateSettings } from "@editor/tools";
import { createInitialAppState } from "@editor/initial-state";
import { resolveGeneratedOutputPlacement } from "./outputPlacement";
describe("generated output placement", () => {
test("centers text-to-image output in the active artboard", () => {
const placement = resolveGeneratedOutputPlacement({
document: document(),
selection: { artboardId: "artboard", layerIds: [] },
settings: settings("text-to-image"),
intrinsicSize: { w: 400, h: 200 },
});
expect(placement).toEqual({
artboardId: "artboard",
layerName: "Generated image",
transform: { position: { x: 310, y: 270 }, scale: { x: 1, y: 1 }, rotation: 0 },
});
});
test("matches image-to-image output to the selected layer bounds", () => {
const placement = resolveGeneratedOutputPlacement({
document: document(),
selection: selected(),
settings: settings("image-to-image"),
intrinsicSize: { w: 200, h: 250 },
});
expect(placement.transform).toEqual({
position: { x: 80, y: 90 },
scale: { x: 0.75, y: 1 },
rotation: 12,
});
});
test("places an inpaint crop at its recorded document bounds", () => {
const placement = resolveGeneratedOutputPlacement({
document: document(),
selection: selected(),
settings: settings("inpaint"),
intrinsicSize: { w: 128, h: 64 },
inpaintBundle: {
inputImage: "input", sourceImage: "source", contextImage: "context", maskImage: "mask", editMaskImage: "edit", blendMaskImage: "blend", width: 256, height: 128,
targetLayerId: "source", regionId: "region", sourceAssetId: "source-asset", maskAssetId: "mask-asset", revision: { source: "source-revision", mask: "mask-revision" },
crop: { assetBounds: { x: 0, y: 0, w: 256, h: 128 }, documentBounds: { x: 140, y: 150, w: 384, h: 64 }, padding: 16, maskedAreaOnly: true },
mask: { polarity: "hidden", activeBounds: { x: 20, y: 20, w: 40, h: 40 } },
placement: { artboardId: "artboard", layerName: "Source inpaint", transform: { position: { x: 140, y: 150 }, scale: { x: 1.5, y: 0.5 }, rotation: 12 } },
backend: { growMaskBy: 6, maskedContent: "neutral", maskBlur: 0, maskFeather: 0, maskExpand: 0, cropPadding: 16 },
},
});
expect(placement.transform).toEqual({ position: { x: 140, y: 150 }, scale: { x: 3, y: 1 }, rotation: 12 });
});
test("extends outpaint output left and top while preserving source pixel scale", () => {
const outpaint = { ...settings("outpaint"), outpaint: { left: 40, top: 20, right: 10, bottom: 30, feathering: 8 } };
const placement = resolveGeneratedOutputPlacement({
document: document(),
selection: selected(),
settings: outpaint,
intrinsicSize: { w: 150, h: 550 },
});
expect(placement.transform).toEqual({
position: { x: 20, y: 80 },
scale: { x: 1.5, y: 0.5 },
rotation: 12,
});
});
});
function settings(mode: GenerateSettings["mode"]): GenerateSettings {
return { ...createInitialAppState("Test").editor.tools.generate, mode };
}
function selected() {
return { artboardId: "artboard", layerIds: ["source"] };
}
function document(): ImageDocument {
return {
id: "document", name: "Test", version: 1, inpaintRegions: [],
assets: [
{ id: "source-asset", name: "Source", mimeType: "image/png", source: "source", intrinsicSize: { w: 100, h: 500 } },
{ id: "mask-asset", name: "Mask", mimeType: "image/png", source: "mask", intrinsicSize: { w: 100, h: 500 } },
],
artboards: [{
id: "artboard", name: "Artboard", bounds: { x: 10, y: 20, w: 1000, h: 700 }, backgroundColor: "transparent", visible: true, locked: false,
layers: [{ id: "source", type: "raster", name: "Source", visible: true, locked: false, opacity: 1, assetId: "source-asset", transform: { position: { x: 80, y: 90 }, scale: { x: 1.5, y: 0.5 }, rotation: 12 } }],
}],
};
}

View File

@@ -0,0 +1,105 @@
import type { ImageDocument } from "@core/document";
import type { Size, Transform } from "@core/geometry";
import type { ArtboardId } from "@core/id";
import { createDocumentReadIndex } from "@editor/document-indexes";
import type { GenerationCandidate, SelectionState } from "@editor/state";
import type { GenerateSettings } from "@editor/tools";
import type { InpaintBundle } from "./inpaintPrep";
export type GeneratedOutputPlacement = GenerationCandidate["placement"];
export function resolveGeneratedOutputPlacement(options: {
document: ImageDocument;
selection: SelectionState;
settings: GenerateSettings;
intrinsicSize: Size;
inpaintBundle?: InpaintBundle;
}): GeneratedOutputPlacement {
const { document, selection, settings, intrinsicSize, inpaintBundle } = options;
if (inpaintBundle) {
return {
artboardId: inpaintBundle.placement.artboardId,
layerName: inpaintBundle.placement.layerName,
transform: scaleInpaintPlacement(inpaintBundle, intrinsicSize),
};
}
const artboard = selection.artboardId
? document.artboards.find((candidate) => candidate.id === selection.artboardId)
: document.artboards[0];
if (!artboard) throw new Error("Create an artboard before placing generated output.");
if (settings.mode === "text-to-image") {
return {
artboardId: artboard.id,
layerName: "Generated image",
transform: {
position: {
x: artboard.bounds.x + (artboard.bounds.w - intrinsicSize.w) / 2,
y: artboard.bounds.y + (artboard.bounds.h - intrinsicSize.h) / 2,
},
scale: { x: 1, y: 1 },
rotation: 0,
},
};
}
const target = resolveTarget(document, selection, artboard.id);
const sourceScale = {
x: target.layer.transform.scale.x * target.asset.intrinsicSize.w / Math.max(1, intrinsicSize.w),
y: target.layer.transform.scale.y * target.asset.intrinsicSize.h / Math.max(1, intrinsicSize.h),
};
if (settings.mode === "image-to-image") {
return {
artboardId: artboard.id,
layerName: "Generated image",
transform: {
position: { ...target.layer.transform.position },
scale: sourceScale,
rotation: target.layer.transform.rotation,
},
};
}
if (settings.mode === "outpaint") {
const pixelScale = target.layer.transform.scale;
return {
artboardId: artboard.id,
layerName: "Generated image",
transform: {
position: {
x: target.layer.transform.position.x - settings.outpaint.left * pixelScale.x,
y: target.layer.transform.position.y - settings.outpaint.top * pixelScale.y,
},
scale: { ...pixelScale },
rotation: target.layer.transform.rotation,
},
};
}
throw new Error("Inpaint output requires an inpaint placement bundle.");
}
function resolveTarget(document: ImageDocument, selection: SelectionState, artboardId: ArtboardId) {
const layerId = selection.layerIds[0];
const index = createDocumentReadIndex(document);
const layerInfo = layerId ? index.layerInfoById.get(layerId) : undefined;
if (!layerInfo || layerInfo.artboardId !== artboardId || (layerInfo.layer.type !== "image" && layerInfo.layer.type !== "raster")) {
throw new Error("Select one image or raster layer before placing generated output.");
}
const asset = index.assetById.get(layerInfo.layer.assetId);
if (!asset) throw new Error("The selected layer is missing its source image.");
return { layer: layerInfo.layer, asset };
}
function scaleInpaintPlacement(bundle: InpaintBundle, intrinsicSize: Size): Transform {
return {
position: { ...bundle.placement.transform.position },
scale: {
x: bundle.placement.transform.scale.x * bundle.width / Math.max(1, intrinsicSize.w),
y: bundle.placement.transform.scale.y * bundle.height / Math.max(1, intrinsicSize.h),
},
rotation: bundle.placement.transform.rotation,
};
}

View File

@@ -0,0 +1,72 @@
import { describe, expect, test } from "bun:test";
import type { ImageDocument } from "@core/document";
import { initialToolState } from "@editor/tools";
import { checkGenerationPreconditions } from "./preconditions";
describe("generation preconditions", () => {
test("allows text-to-image with a prompt and artboard but no layer selection", () => {
expect(checkGenerationPreconditions(document(), { artboardId: "artboard", layerIds: [] }, settings("text-to-image"))).toEqual({ ready: true });
});
test("requires exactly one source layer for image-to-image and outpaint", () => {
const selection = { artboardId: "artboard", layerIds: [] };
expect(checkGenerationPreconditions(document(), selection, settings("image-to-image"))).toEqual({ ready: false, message: "Select exactly one image or raster layer for image-to-image generation." });
expect(checkGenerationPreconditions(document(), selection, settings("outpaint"))).toEqual({ ready: false, message: "Select exactly one image or raster layer to outpaint." });
});
test("requires an enabled mask for inpaint", () => {
expect(checkGenerationPreconditions(document(), selected(), settings("inpaint"))).toEqual({ ready: false, message: "Paint an AI edit region over the area you want to replace.", repair: "add-mask" });
});
test("allows inpaint when the selected source has an aligned enabled mask", () => {
expect(checkGenerationPreconditions(document(true), selected(), settings("inpaint"))).toEqual({ ready: true });
});
test("explains prompt and outpaint-padding requirements", () => {
expect(checkGenerationPreconditions(document(), selected(), { ...settings("image-to-image"), prompt: " " })).toEqual({ ready: false, message: "Enter a prompt to generate an image." });
expect(checkGenerationPreconditions(document(), selected(), { ...settings("outpaint"), outpaint: { left: 0, top: 0, right: 0, bottom: 0, feathering: 0 } })).toEqual({ ready: false, message: "Add outpaint padding on at least one side.", repair: "set-outpaint-padding" });
});
});
function settings(mode: "text-to-image" | "image-to-image" | "inpaint" | "outpaint") {
return { ...initialToolState.generate, mode, prompt: "A lighthouse" };
}
function selected() {
return { artboardId: "artboard" as const, layerIds: ["source"] };
}
function document(masked = false): ImageDocument {
const transform = { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 };
return {
id: "document",
name: "Test",
version: 1,
assets: [
{ id: "source-asset", name: "Source", mimeType: "image/png", source: "source", intrinsicSize: { w: 100, h: 100 } },
{ id: "mask-asset", name: "Mask", mimeType: "image/png", source: "mask", intrinsicSize: { w: 100, h: 100 } },
],
inpaintRegions: masked ? [{ id: "region", name: "AI edit", targetLayerId: "source", maskAssetId: "mask-asset", enabled: true }] : [],
artboards: [{
id: "artboard",
name: "Artboard",
bounds: { x: 0, y: 0, w: 100, h: 100 },
backgroundColor: "transparent",
visible: true,
locked: false,
layers: [
{ id: "mask", type: "raster", name: "Mask", visible: true, locked: false, opacity: 1, assetId: "mask-asset", transform },
{
id: "source",
type: "raster",
name: "Source",
visible: true,
locked: false,
opacity: 1,
assetId: "source-asset",
transform,
},
],
}],
};
}

View File

@@ -0,0 +1,67 @@
import type { ImageDocument } from "@core/document";
import { createDocumentReadIndex, resolveIndexedLayerBounds } from "@editor/document-indexes";
import type { SelectionState } from "@editor/state";
import type { GenerateSettings } from "@editor/tools";
export type GenerationPrecondition =
| { ready: true }
| { ready: false; message: string; repair?: "add-mask" | "set-outpaint-padding" };
export function checkGenerationPreconditions(
document: ImageDocument,
selection: SelectionState,
settings: GenerateSettings,
): GenerationPrecondition {
if (!settings.prompt.trim()) return missing("Enter a prompt to generate an image.");
const artboard = selection.artboardId
? document.artboards.find((candidate) => candidate.id === selection.artboardId)
: document.artboards[0];
if (!artboard) return missing("Create an artboard before generating an image.");
if (settings.mode === "text-to-image") return { ready: true };
if (selection.layerIds.length !== 1 || !selection.layerIds[0]) {
return missing(modeSelectionMessage(settings.mode));
}
const index = createDocumentReadIndex(document);
const layerInfo = index.layerInfoById.get(selection.layerIds[0]);
if (!layerInfo || layerInfo.artboardId !== artboard.id || (layerInfo.layer.type !== "image" && layerInfo.layer.type !== "raster")) {
return missing(modeSelectionMessage(settings.mode));
}
const asset = index.assetById.get(layerInfo.layer.assetId);
if (!asset) return missing("The selected layer is missing its source image.");
if (asset.intrinsicSize.w <= 0 || asset.intrinsicSize.h <= 0) return missing("The selected image has an invalid size.");
if (settings.mode === "outpaint") {
const padding = settings.outpaint;
if (padding.left + padding.top + padding.right + padding.bottom <= 0) {
return missing("Add outpaint padding on at least one side.", "set-outpaint-padding");
}
}
if (settings.mode !== "inpaint") return { ready: true };
const region = document.inpaintRegions.find((candidate) => candidate.targetLayerId === layerInfo.layer.id && candidate.enabled);
if (!region) return missing("Paint an AI edit region over the area you want to replace.", "add-mask");
const maskAsset = index.assetById.get(region.maskAssetId);
if (!maskAsset) return missing("The AI edit region is missing its mask data.");
if (Math.round(asset.intrinsicSize.w) !== Math.round(maskAsset.intrinsicSize.w) || Math.round(asset.intrinsicSize.h) !== Math.round(maskAsset.intrinsicSize.h)) {
return missing("The selected layer and mask image sizes must match.");
}
if (!resolveIndexedLayerBounds(index, layerInfo.layer)) return missing("The selected layer has invalid geometry.");
return { ready: true };
}
function modeSelectionMessage(mode: GenerateSettings["mode"]): string {
if (mode === "image-to-image") return "Select exactly one image or raster layer for image-to-image generation.";
if (mode === "inpaint") return "Select exactly one masked image or raster layer to inpaint.";
return "Select exactly one image or raster layer to outpaint.";
}
function missing(message: string, repair?: Extract<GenerationPrecondition, { ready: false }>["repair"]): GenerationPrecondition {
return { ready: false, message, ...(repair ? { repair } : {}) };
}

View File

@@ -1,6 +1,5 @@
import { commandIds } from "@commands/ids";
import type { ImageDocument } from "@core/document";
import type { Transform } from "@core/geometry";
import type { Layer } from "@core/layer";
import { getLayerMask } from "@core/layer-mask-utils";
import type { AppStore } from "@editor/store";
@@ -9,6 +8,8 @@ import type { GenerateSettings } from "@editor/tools";
import { buildInpaintBundle, type InpaintBundle } from "./inpaintPrep";
import { imageSourceToDataUrl, loadImageSize } from "@platform/browser/imageRaster";
import { requestGeneration } from "@platform/comfy/generationClient";
import { checkGenerationPreconditions } from "./preconditions";
import { resolveGeneratedOutputPlacement } from "./outputPlacement";
export async function runGenerate(options: {
document: ImageDocument;
@@ -16,10 +17,14 @@ export async function runGenerate(options: {
viewport: ViewportState;
settings: GenerateSettings;
dispatch: AppStore["dispatch"];
signal?: AbortSignal;
onProgress?: (progress: number, detail: string) => void;
}) {
const { document, selection, settings, dispatch } = options;
const precondition = checkGenerationPreconditions(document, selection, settings);
if (!precondition.ready) throw new Error(precondition.message);
const artboard = selection.artboardId ? document.artboards.find((candidate) => candidate.id === selection.artboardId) : document.artboards[0];
if (!artboard) return;
if (!artboard) throw new Error("Create an artboard before generating an image.");
const target = resolveSelectedImage(document, selection);
const inpaintBundle = settings.mode === "inpaint" ? await buildInpaintBundle(document, selection, settings) : undefined;
@@ -36,22 +41,19 @@ export async function runGenerate(options: {
inputImage,
maskImage,
inpaintBundle,
signal: options.signal,
onProgress: options.onProgress,
});
const intrinsicSize = await loadImageSize(generated.source);
const targetArtboardId = inpaintBundle?.placement.artboardId ?? artboard.id;
const placement = {
artboardId: targetArtboardId,
layerName: inpaintBundle?.placement.layerName ?? "Generated image",
transform: generatedLayerTransform(inpaintBundle, intrinsicSize),
};
for (const result of generated.results) {
const intrinsicSize = await loadImageSize(result.source);
const placement = resolveGeneratedOutputPlacement({ document, selection, settings, intrinsicSize, inpaintBundle });
dispatch(commandIds.generationAddCandidate, {
candidate: createGenerationCandidate({
source: generated.source,
mimeType: generated.mimeType,
source: result.source,
mimeType: result.mimeType,
intrinsicSize,
settings: requestSettings,
seed,
seed: result.seed || seed,
width,
height,
inputImage,
@@ -60,12 +62,15 @@ export async function runGenerate(options: {
inpaintBundle,
}),
});
}
}
export async function runGenerateFromCandidate(options: {
candidate: GenerationCandidate;
settings?: GenerateSettings;
dispatch: AppStore["dispatch"];
signal?: AbortSignal;
onProgress?: (progress: number, detail: string) => void;
}) {
const settings = options.settings ?? options.candidate.settings;
const seed = resolveSeed(settings.seed);
@@ -77,20 +82,23 @@ export async function runGenerateFromCandidate(options: {
inputImage: options.candidate.inputImage,
maskImage: options.candidate.maskImage,
inpaintCandidate: options.candidate,
signal: options.signal,
onProgress: options.onProgress,
});
const intrinsicSize = await loadImageSize(generated.source);
for (const result of generated.results) {
const intrinsicSize = await loadImageSize(result.source);
options.dispatch(commandIds.generationAddCandidate, {
candidate: {
...options.candidate,
id: crypto.randomUUID(),
source: generated.source,
mimeType: generated.mimeType,
source: result.source,
mimeType: result.mimeType,
intrinsicSize,
settings: requestSettings,
seed,
seed: result.seed || seed,
},
});
}
}
function createGenerationCandidate(options: {
@@ -118,15 +126,19 @@ function createGenerationCandidate(options: {
height: options.height,
inputImage: options.inputImage,
maskImage: options.maskImage,
blendMaskImage: options.inpaintBundle?.blendMaskImage,
placement: options.placement,
inpaint: options.inpaintBundle
? {
targetLayerId: options.inpaintBundle.targetLayerId,
maskLayerId: options.inpaintBundle.maskLayerId,
regionId: options.inpaintBundle.regionId,
sourceAssetId: options.inpaintBundle.sourceAssetId,
maskAssetId: options.inpaintBundle.maskAssetId,
inputImage: options.inpaintBundle.inputImage,
maskImage: options.inpaintBundle.maskImage,
editMaskImage: options.inpaintBundle.editMaskImage,
blendMaskImage: options.inpaintBundle.blendMaskImage,
revision: options.inpaintBundle.revision,
crop: options.inpaintBundle.crop,
mask: options.inpaintBundle.mask,
backend: options.inpaintBundle.backend,
@@ -143,6 +155,8 @@ async function requestGenerate(options: {
maskImage?: string;
inpaintBundle?: InpaintBundle;
inpaintCandidate?: GenerationCandidate;
signal?: AbortSignal;
onProgress?: (progress: number, detail: string) => void;
}) {
return requestGeneration({
architecture: options.settings.architecture,
@@ -160,11 +174,14 @@ async function requestGenerate(options: {
scheduler: options.settings.scheduler,
width: options.width,
height: options.height,
batchSize: options.settings.batchSize,
refinePass: options.settings.refinePass,
refineStrength: options.settings.refineStrength,
outpaint: options.settings.outpaint,
inpaint: resolveInpaintRequest(options.inpaintBundle, options.inpaintCandidate, options.settings),
inputImage: options.inputImage,
maskImage: options.maskImage,
});
}, options.signal, options.onProgress);
}
function resolveInpaintRequest(inpaintBundle: InpaintBundle | undefined, inpaintCandidate: GenerationCandidate | undefined, settings: GenerateSettings) {
@@ -179,6 +196,9 @@ function resolveInpaintRequest(inpaintBundle: InpaintBundle | undefined, inpaint
maskPolarity: inpaintBundle.mask.polarity,
crop: inpaintBundle.crop,
placement: inpaintBundle.placement,
structureControl: settings.inpaint.structureControl,
controlStrength: settings.inpaint.controlStrength,
controlModel: settings.inpaint.controlModel,
};
}
@@ -193,24 +213,15 @@ function resolveInpaintRequest(inpaintBundle: InpaintBundle | undefined, inpaint
maskPolarity: inpaintCandidate.inpaint.mask.polarity,
crop: inpaintCandidate.inpaint.crop,
placement: inpaintCandidate.placement,
structureControl: settings.inpaint.structureControl,
controlStrength: settings.inpaint.controlStrength,
controlModel: settings.inpaint.controlModel,
};
}
return settings.inpaint;
}
function generatedLayerTransform(inpaintBundle: InpaintBundle | undefined, intrinsicSize: { w: number; h: number }): Transform {
if (!inpaintBundle) return { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 };
return {
position: { ...inpaintBundle.placement.transform.position },
scale: {
x: (inpaintBundle.placement.transform.scale.x * inpaintBundle.width) / Math.max(1, intrinsicSize.w),
y: (inpaintBundle.placement.transform.scale.y * inpaintBundle.height) / Math.max(1, intrinsicSize.h),
},
rotation: inpaintBundle.placement.transform.rotation,
};
}
function resolveSeed(seed: number): number {
return seed < 0 ? Math.floor(Math.random() * 2 ** 32) : Math.round(seed);
}
@@ -219,11 +230,11 @@ function resolveSelectedImage(document: ImageDocument, selection: SelectionState
const layerId = selection.layerIds[0];
if (!layerId) return undefined;
const layer = findLayer(document.artboards.find((artboard) => artboard.id === selection.artboardId)?.layers ?? [], layerId);
if (!layer || layer.type === "group") return undefined;
if (!layer || (layer.type !== "image" && layer.type !== "raster")) return undefined;
const asset = document.assets.find((candidate) => candidate.id === layer.assetId);
const layerMask = getLayerMask(layer);
const maskLayer = layerMask?.enabled ? findLayer(document.artboards.flatMap((artboard) => artboard.layers), layerMask.maskLayerId) : undefined;
const maskAsset = maskLayer && maskLayer.type !== "group" ? document.assets.find((candidate) => candidate.id === maskLayer.assetId) : undefined;
const maskAsset = maskLayer && (maskLayer.type === "image" || maskLayer.type === "raster") ? document.assets.find((candidate) => candidate.id === maskLayer.assetId) : undefined;
return asset ? { layer, asset, maskAsset } : undefined;
}

View File

@@ -0,0 +1,125 @@
import { describe, expect, test } from "bun:test";
import { documentCommands } from "@commands/document";
import { inpaintRegionCommands } from "@commands/inpaint-region";
import { generationCommands } from "@commands/generation";
import { commandIds } from "@commands/ids";
import { createCommandRegistry } from "@commands/registry";
import { toolCommands } from "@commands/tool";
import { createInitialAppState } from "@editor/initial-state";
import { createAppStore } from "@editor/store";
import { initialToolState } from "@editor/tools";
import type { GenerationCandidate } from "@editor/state";
import { createGenerationWorkflow, type GenerationWorkflowDependencies } from "./workflow";
describe("generation workflow", () => {
test("reads canonical application state when generation starts", async () => {
const app = createTestApp();
let prompt = "";
const workflow = createGenerationWorkflow(app.store, dependencies({
runGenerate: async (options) => { prompt = options.settings.prompt; },
}));
app.store.dispatch(commandIds.toolSetGenerateSettings, { prompt: "Latest prompt" });
await workflow.generate();
expect(prompt).toBe("Latest prompt");
expect(app.store.getState().editor.generation.jobs[0]?.status).toBe("succeeded");
});
test("uses one acceptance path to add a candidate as a layer", () => {
const app = createTestApp();
const artboardId = app.store.getState().document.artboards[0]?.id;
if (!artboardId) throw new Error("Expected default artboard");
app.store.dispatch(commandIds.generationAddCandidate, { candidate: candidate(artboardId) });
const ids = ["asset-new", "layer-new"];
const workflow = createGenerationWorkflow(app.store, dependencies({ createId: () => ids.shift() ?? "unused" }));
workflow.applyCandidateAsLayer("candidate");
expect(app.store.getState().document.assets.some((asset) => asset.id === "asset-new")).toBe(true);
expect(app.store.getState().document.artboards[0]?.layers.some((layer) => layer.id === "layer-new")).toBe(true);
expect(app.store.getState().editor.generation.candidates).toHaveLength(0);
});
test("repairs replace prerequisites by adding an aligned editable mask through commands", async () => {
const app = createTestApp();
const state = app.store.getState();
state.document.assets.push({ id: "source-asset", name: "Source", mimeType: "image/png", source: "source", intrinsicSize: { w: 80, h: 60 } });
state.document.artboards[0]!.layers.push({ id: "source", type: "raster", name: "Source", visible: true, locked: false, opacity: 1, assetId: "source-asset", transform: { position: { x: 4, y: 8 }, scale: { x: 2, y: 2 }, rotation: 0 } });
state.editor.selection = { artboardId: "artboard", layerIds: ["source"] };
const ids = ["mask-asset", "region"];
const workflow = createGenerationWorkflow(app.store, dependencies({ createId: () => ids.shift() ?? "unused" }));
await workflow.prepareInpaintMask();
const next = app.store.getState();
expect(next.document.inpaintRegions).toContainEqual({ id: "region", name: "Source AI edit", targetLayerId: "source", maskAssetId: "mask-asset", enabled: true });
expect(next.editor.maskEdit).toEqual({ kind: "inpaintRegion", targetLayerId: "source", inpaintRegionId: "region", maskAssetId: "mask-asset", viewMode: "overlay" });
});
test("cancels the active operation and records a cancelled job", async () => {
const app = createTestApp();
let receivedSignal: AbortSignal | undefined;
const workflow = createGenerationWorkflow(app.store, dependencies({
runGenerate: async (options) => {
receivedSignal = options.signal;
await new Promise<void>((_resolve, reject) => options.signal?.addEventListener("abort", () => reject(new DOMException("Cancelled", "AbortError")), { once: true }));
},
}));
const running = workflow.generate();
await Promise.resolve();
workflow.cancel();
await running;
expect(receivedSignal?.aborted).toBe(true);
expect(app.store.getState().editor.generation.jobs[0]?.status).toBe("cancelled");
});
});
function dependencies(overrides: Partial<GenerationWorkflowDependencies>): GenerationWorkflowDependencies {
return {
runGenerate: async () => undefined,
runGenerateFromCandidate: async () => undefined,
createMaskedPixelReplacementSource: async () => "replacement",
createRefinementMask: async () => "mask",
createInpaintRegionMask: async () => "mask",
loadGenerationResources: async () => undefined,
createId: () => crypto.randomUUID(),
...overrides,
};
}
function candidate(artboardId: string): GenerationCandidate {
return {
id: "candidate",
source: "generated",
mimeType: "image/png",
intrinsicSize: { w: 64, h: 64 },
mode: "text-to-image",
settings: initialToolState.generate,
seed: 1,
width: 64,
height: 64,
placement: {
artboardId,
layerName: "Generated",
transform: { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 },
},
};
}
function createTestApp() {
const state = createInitialAppState("Test");
state.document.artboards.push({
id: "artboard",
name: "Artboard",
bounds: { x: 0, y: 0, w: 100, h: 100 },
backgroundColor: "transparent",
visible: true,
locked: false,
layers: [],
});
const registry = createCommandRegistry([...documentCommands, ...inpaintRegionCommands, ...toolCommands, ...generationCommands]);
return { store: createAppStore(state, registry) };
}

View File

@@ -0,0 +1,194 @@
import { commandIds } from "@commands/ids";
import type { Layer } from "@core/layer";
import type { GenerationCandidate, GenerationJobKind } from "@editor/state";
import type { AppStore } from "@editor/store";
import type { GenerateSettings } from "@editor/tools";
import { createInpaintRegionMask, createRefinementMask } from "@operations/masks/rasterActions";
import { createMaskedPixelReplacementSource } from "./candidateActions";
import { runGenerationJob } from "./generationJob";
import { loadGenerationResources } from "./loadResources";
import { runGenerate, runGenerateFromCandidate } from "./runGenerate";
import { checkGenerationPreconditions } from "./preconditions";
export type GenerationWorkflow = ReturnType<typeof createGenerationWorkflow>;
export type GenerationWorkflowDependencies = {
runGenerate: typeof runGenerate;
runGenerateFromCandidate: typeof runGenerateFromCandidate;
createMaskedPixelReplacementSource: typeof createMaskedPixelReplacementSource;
createRefinementMask: typeof createRefinementMask;
createInpaintRegionMask: typeof createInpaintRegionMask;
loadGenerationResources: typeof loadGenerationResources;
createId(): string;
};
const defaultDependencies: GenerationWorkflowDependencies = {
runGenerate,
runGenerateFromCandidate,
createMaskedPixelReplacementSource,
createRefinementMask,
createInpaintRegionMask,
loadGenerationResources,
createId: () => crypto.randomUUID(),
};
export function createGenerationWorkflow(store: AppStore, dependencies: GenerationWorkflowDependencies = defaultDependencies) {
let activeController: AbortController | undefined;
const job = async (kind: GenerationJobKind, label: string, task: (signal: AbortSignal, report: (progress: number, detail: string) => void) => Promise<void>) => {
if (store.getState().editor.generation.jobs.some((candidate) => candidate.status === "running")) return;
const controller = new AbortController();
activeController = controller;
try {
await runGenerationJob({ kind, label, dispatch: store.dispatch, signal: controller.signal, task });
} finally {
if (activeController === controller) activeController = undefined;
}
};
return {
precondition: () => {
const state = store.getState();
return checkGenerationPreconditions(state.document, state.editor.selection, state.editor.tools.generate);
},
loadResources: () => dependencies.loadGenerationResources(store),
prepareInpaintMask: async () => {
const state = store.getState();
const layerId = state.editor.selection.layerIds.length === 1 ? state.editor.selection.layerIds[0] : undefined;
if (!layerId) return;
const artboard = state.document.artboards.find((candidate) => candidate.id === state.editor.selection.artboardId);
const layer = artboard ? findLayer(artboard.layers, layerId) : undefined;
if (!layer || (layer.type !== "image" && layer.type !== "raster")) return;
const asset = state.document.assets.find((candidate) => candidate.id === layer.assetId);
if (!asset) return;
const existing = state.document.inpaintRegions.find((region) => region.targetLayerId === layerId && region.enabled);
if (existing) {
store.dispatch(commandIds.toolEnterInpaintRegionEdit, { targetLayerId: layerId, regionId: existing.id });
return;
}
const source = await dependencies.createInpaintRegionMask(asset.intrinsicSize.w, asset.intrinsicSize.h);
const maskAssetId = dependencies.createId();
const regionId = dependencies.createId();
store.dispatch(commandIds.documentAddInpaintRegion, {
region: { id: regionId, name: `${layer.name} AI edit`, targetLayerId: layer.id, maskAssetId, enabled: true },
maskAsset: { id: maskAssetId, name: `${layer.name} AI edit mask`, mimeType: "image/png", source, intrinsicSize: { ...asset.intrinsicSize } },
});
store.dispatch(commandIds.toolEnterInpaintRegionEdit, { targetLayerId: layer.id, regionId });
},
generate: () => job("generate", "Generating", async (signal, report) => {
const state = store.getState();
await dependencies.runGenerate({
document: state.document,
selection: state.editor.selection,
viewport: state.editor.viewport,
settings: state.editor.tools.generate,
dispatch: store.dispatch,
signal,
onProgress: report,
});
}),
regenerate: (candidateId: string, settings?: GenerateSettings, label = "Regenerate") =>
job("regenerate", label, async (signal, report) => {
const candidate = findCandidate(store, candidateId);
const nextSettings = settings ?? candidate.settings;
store.dispatch(commandIds.toolSetGenerateSettings, nextSettings);
await dependencies.runGenerateFromCandidate({ candidate, settings: nextSettings, dispatch: store.dispatch, signal, onProgress: report });
}),
rebuildFromCurrentRegion: (candidateId: string) =>
job("regenerate", "Rebuilding from current edit region", async (signal, report) => {
const candidate = findCandidate(store, candidateId);
if (!candidate.inpaint) throw new Error("Only inpaint candidates can rebuild from an edit region.");
store.dispatch(commandIds.selectionSet, { artboardId: candidate.placement.artboardId, layerIds: [candidate.inpaint.targetLayerId] });
store.dispatch(commandIds.toolSetGenerateSettings, candidate.settings);
const state = store.getState();
await dependencies.runGenerate({
document: state.document,
selection: state.editor.selection,
viewport: state.editor.viewport,
settings: candidate.settings,
dispatch: store.dispatch,
signal,
onProgress: report,
});
}),
applyCandidateAsLayer: (candidateId: string) => {
store.dispatch(commandIds.generationApplyCandidateAsLayer, {
candidateId,
assetId: dependencies.createId(),
layerId: dependencies.createId(),
});
},
applyCandidateAsRefinementLayer: (candidateId: string) =>
job("refine", "Adding refinement mask", async () => {
const candidate = findCandidate(store, candidateId);
const layerId = dependencies.createId();
const maskAssetId = dependencies.createId();
const width = Math.max(1, Math.round(candidate.intrinsicSize.w));
const height = Math.max(1, Math.round(candidate.intrinsicSize.h));
const source = await dependencies.createRefinementMask(width, height);
store.dispatch(commandIds.generationApplyCandidateAsLayer, {
candidateId,
assetId: dependencies.createId(),
layerId,
});
store.dispatch(commandIds.documentAddLayerMask, {
layerId,
asset: {
id: maskAssetId,
name: `${candidate.placement.layerName} refinement mask`,
mimeType: "image/png",
source,
intrinsicSize: { w: width, h: height },
},
maskLayer: {
id: dependencies.createId(),
type: "raster",
name: `${candidate.placement.layerName} refinement mask`,
visible: true,
locked: false,
opacity: 1,
assetId: maskAssetId,
transform: {
position: { ...candidate.placement.transform.position },
scale: { ...candidate.placement.transform.scale },
rotation: candidate.placement.transform.rotation,
},
},
});
store.dispatch(commandIds.toolSetActive, { tool: "eraser" });
}),
replaceCandidatePixels: (candidateId: string) =>
job("replace", "Replacing pixels", async () => {
const candidate = findCandidate(store, candidateId);
const source = await dependencies.createMaskedPixelReplacementSource(store.getState().document, candidate);
store.dispatch(commandIds.generationReplaceCandidatePixels, { candidateId, source, mimeType: "image/png" });
}),
cancel: () => activeController?.abort(),
};
}
function findLayer(layers: readonly Layer[], layerId: string): Layer | undefined {
for (const layer of layers) {
if (layer.id === layerId) return layer;
if (layer.type === "group") {
const child = findLayer(layer.children, layerId);
if (child) return child;
}
}
return undefined;
}
function findCandidate(store: AppStore, candidateId: string): GenerationCandidate {
const candidate = store.getState().editor.generation.candidates.find((item) => item.id === candidateId);
if (!candidate) throw new Error("This generation candidate is no longer available.");
return candidate;
}

View File

@@ -14,20 +14,21 @@ export function resolveChromaKeyTarget(document: ImageDocument, selection: Selec
const layerId = selection.layerIds[0];
if (selection.layerIds.length !== 1 || !layerId) return undefined;
const layer = findLayer(document.artboards.find((artboard) => artboard.id === selection.artboardId)?.layers ?? [], layerId);
if (!layer || layer.type === "group") return undefined;
if (!layer || (layer.type !== "image" && layer.type !== "raster")) return undefined;
const asset = document.assets.find((candidate) => candidate.id === layer.assetId);
const bounds = resolveTransformTargetBounds(document, { type: "layer", id: layer.id });
const layerMask = getLayerMask(layer);
const maskLayer = layerMask?.enabled ? findLayer(document.artboards.flatMap((artboard) => artboard.layers), layerMask.maskLayerId) : undefined;
const maskAsset = maskLayer && maskLayer.type !== "group" ? document.assets.find((candidate) => candidate.id === maskLayer.assetId) : undefined;
const maskAsset = maskLayer && (maskLayer.type === "image" || maskLayer.type === "raster") ? document.assets.find((candidate) => candidate.id === maskLayer.assetId) : undefined;
return asset && bounds ? { layer, asset, bounds, maskLayer, maskAsset } : undefined;
}
export async function applyChromaKeyMask(target: NonNullable<ReturnType<typeof resolveChromaKeyTarget>>, settings: ChromaKeySettings, dispatch: AppStore["dispatch"]) {
const source = await createChromaKeyMask(target.asset.source, target.asset.intrinsicSize.w, target.asset.intrinsicSize.h, settings);
dispatch(commandIds.toolSetBrushStrokePreview, undefined);
if (target.maskAsset && target.maskLayer && target.maskLayer.type !== "group") {
if (target.maskAsset && target.maskLayer && (target.maskLayer.type === "image" || target.maskLayer.type === "raster")) {
dispatch(commandIds.documentApplyLayerMaskOperation, { maskLayerId: target.maskLayer.id, source, mimeType: "image/png", operation: { type: "chromaKey" } });
dispatch(commandIds.workspaceSetPanel, { panel: "none" });
return;
}
const assetId = crypto.randomUUID();
@@ -40,7 +41,7 @@ export async function applyChromaKeyMask(target: NonNullable<ReturnType<typeof r
maskLayer: { id: maskLayerId, type: "raster", name: `${target.layer.name} Chroma Mask`, visible: true, locked: false, opacity: 1, assetId, transform: { position: { x: target.bounds.x, y: target.bounds.y }, scale: { x: target.bounds.w / width, y: target.bounds.h / height }, rotation: target.layer.transform.rotation } },
});
dispatch(commandIds.toolExitMaskEdit, undefined);
dispatch(commandIds.toolSetActive, { tool: "chromaKey" });
dispatch(commandIds.workspaceSetPanel, { panel: "none" });
}
function findLayer(layers: readonly Layer[], layerId: string): Layer | undefined {

52
operations/masks/lasso.ts Normal file
View File

@@ -0,0 +1,52 @@
import { commandIds } from "@commands/ids";
import type { Vec2D } from "@core/geometry";
import type { Layer } from "@core/layer";
import type { AppStore } from "@editor/store";
import { applyPolygonMask } from "@platform/browser/maskRaster";
export async function commitInpaintLasso(store: AppStore) {
const state = store.getState();
const edit = state.editor.maskEdit;
const session = state.editor.maskShapeSession;
const minimumPoints = session?.shape === "rectangle" ? 2 : 3;
if (edit?.kind !== "inpaintRegion" || !edit.inpaintRegionId || !session || session.points.length < minimumPoints) {
store.dispatch(commandIds.toolClearMaskShape, undefined);
return;
}
const region = state.document.inpaintRegions.find((candidate) => candidate.id === edit.inpaintRegionId);
const target = findLayer(state.document.artboards.flatMap((artboard) => artboard.layers), edit.targetLayerId);
const asset = region ? state.document.assets.find((candidate) => candidate.id === region.maskAssetId) : undefined;
if (!region || !target || (target.type !== "image" && target.type !== "raster") || !asset) {
store.dispatch(commandIds.toolClearMaskShape, undefined);
return;
}
const documentPoints = session.shape === "rectangle" ? rectanglePoints(session.points[0]!, session.points[1]!) : session.points;
const points = documentPoints.map((point) => documentPointToAssetPoint(point, target, asset.intrinsicSize));
const source = await applyPolygonMask(asset.source, asset.intrinsicSize.w, asset.intrinsicSize.h, points, session.mode);
store.dispatch(commandIds.documentApplyInpaintRegionMaskOperation, { regionId: region.id, source, mimeType: "image/png", operation: { type: "paint" } });
store.dispatch(commandIds.toolClearMaskShape, undefined);
}
function rectanglePoints(start: Vec2D, end: Vec2D): Vec2D[] {
return [start, { x: end.x, y: start.y }, end, { x: start.x, y: end.y }];
}
function documentPointToAssetPoint(point: Vec2D, layer: Extract<Layer, { type: "image" | "raster" }>, intrinsicSize: { w: number; h: number }): Vec2D {
const source = layer.sourceRect ?? { x: 0, y: 0, ...intrinsicSize };
const destination = { x: layer.transform.position.x + source.x * layer.transform.scale.x, y: layer.transform.position.y + source.y * layer.transform.scale.y, w: source.w * layer.transform.scale.x, h: source.h * layer.transform.scale.y };
const center = { x: destination.x + destination.w / 2, y: destination.y + destination.h / 2 };
const dx = point.x - center.x;
const dy = point.y - center.y;
const cos = Math.cos(-layer.transform.rotation);
const sin = Math.sin(-layer.transform.rotation);
const x = center.x + dx * cos - dy * sin;
const y = center.y + dx * sin + dy * cos;
return { x: source.x + (x - destination.x) / Math.max(0.0001, layer.transform.scale.x), y: source.y + (y - destination.y) / Math.max(0.0001, layer.transform.scale.y) };
}
function findLayer(layers: readonly Layer[], id: string): Layer | undefined {
for (const layer of layers) {
if (layer.id === id) return layer;
if (layer.type === "group") { const child = findLayer(layer.children, id); if (child) return child; }
}
}

View File

@@ -11,13 +11,18 @@ import { createWandMask } from "@platform/browser/magicWandRaster";
export async function applyMagicWandAt(store: AppStore, point: Vec2D, modeOverride?: EditorState["tools"]["magicWand"]["mode"]) {
const state = store.getState();
if (state.editor.tools.activeTool !== "magicWand") return false;
const target = resolveTarget(state.document, state.editor);
const target = resolveMaskSelectionTarget(state.document, state.editor);
if (!target) return true;
const x = Math.floor((point.x - target.layer.transform.position.x) / Math.max(0.0001, target.layer.transform.scale.x));
const y = Math.floor((point.y - target.layer.transform.position.y) / Math.max(0.0001, target.layer.transform.scale.y));
const assetPoint = documentPointToAssetPoint(point, target.layer, target.asset.intrinsicSize);
const x = Math.floor(assetPoint.x);
const y = Math.floor(assetPoint.y);
if (x < 0 || y < 0 || x >= target.asset.intrinsicSize.w || y >= target.asset.intrinsicSize.h) return true;
const source = await createWandMask(target.asset.source, target.maskAsset?.source, Math.round(target.asset.intrinsicSize.w), Math.round(target.asset.intrinsicSize.h), x, y, { ...state.editor.tools.magicWand, mode: modeOverride ?? state.editor.tools.magicWand.mode });
if (target.maskAsset && target.maskLayer && target.maskLayer.type !== "group") {
const source = await createWandMask(target.asset.source, target.maskAsset?.source, Math.round(target.asset.intrinsicSize.w), Math.round(target.asset.intrinsicSize.h), x, y, { ...state.editor.tools.magicWand, mode: modeOverride ?? state.editor.tools.magicWand.mode, target: target.inpaintRegion ? "inpaint" : "visibility" });
if (target.inpaintRegion && target.maskAsset) {
store.dispatch(commandIds.documentApplyInpaintRegionMaskOperation, { regionId: target.inpaintRegion.id, source, mimeType: "image/png", operation: { type: "magicWand" } });
return true;
}
if (target.maskAsset && target.maskLayer && (target.maskLayer.type === "image" || target.maskLayer.type === "raster")) {
store.dispatch(commandIds.documentApplyLayerMaskOperation, { maskLayerId: target.maskLayer.id, source, mimeType: "image/png", operation: { type: "magicWand" } });
return true;
}
@@ -35,17 +40,31 @@ export async function applyMagicWandAt(store: AppStore, point: Vec2D, modeOverri
return true;
}
function resolveTarget(document: ImageDocument, editor: EditorState) {
export function resolveMaskSelectionTarget(document: ImageDocument, editor: EditorState) {
const layerId = editor.selection.layerIds[0];
if (!layerId || editor.selection.layerIds.length !== 1) return undefined;
const layer = findLayer(document.artboards.flatMap((artboard) => artboard.layers), layerId);
if (!layer || layer.type === "group") return undefined;
if (!layer || (layer.type !== "image" && layer.type !== "raster")) return undefined;
const asset = document.assets.find((candidate) => candidate.id === layer.assetId);
const bounds = resolveTransformTargetBounds(document, { type: "layer", id: layer.id });
const layerMask = getLayerMask(layer);
const inpaintRegion = editor.maskEdit?.kind === "inpaintRegion" ? document.inpaintRegions.find((candidate) => candidate.id === editor.maskEdit?.inpaintRegionId && candidate.targetLayerId === layer.id) : undefined;
const maskLayer = layerMask?.enabled ? findLayer(document.artboards.flatMap((artboard) => artboard.layers), layerMask.maskLayerId) : undefined;
const maskAsset = maskLayer && maskLayer.type !== "group" ? document.assets.find((candidate) => candidate.id === maskLayer.assetId) : undefined;
return asset && bounds ? { layer, asset, bounds, maskLayer, maskAsset } : undefined;
const maskAsset = inpaintRegion ? document.assets.find((candidate) => candidate.id === inpaintRegion.maskAssetId) : maskLayer && (maskLayer.type === "image" || maskLayer.type === "raster") ? document.assets.find((candidate) => candidate.id === maskLayer.assetId) : undefined;
return asset && bounds ? { layer, asset, bounds, maskLayer, maskAsset, inpaintRegion } : undefined;
}
export function documentPointToAssetPoint(point: Vec2D, layer: Extract<Layer, { type: "image" | "raster" }>, intrinsicSize: { w: number; h: number }): Vec2D {
const source = layer.sourceRect ?? { x: 0, y: 0, ...intrinsicSize };
const destination = { x: layer.transform.position.x + source.x * layer.transform.scale.x, y: layer.transform.position.y + source.y * layer.transform.scale.y, w: source.w * layer.transform.scale.x, h: source.h * layer.transform.scale.y };
const center = { x: destination.x + destination.w / 2, y: destination.y + destination.h / 2 };
const dx = point.x - center.x;
const dy = point.y - center.y;
const cos = Math.cos(-layer.transform.rotation);
const sin = Math.sin(-layer.transform.rotation);
const x = center.x + dx * cos - dy * sin;
const y = center.y + dx * sin + dy * cos;
return { x: source.x + (x - destination.x) / Math.max(0.0001, layer.transform.scale.x), y: source.y + (y - destination.y) / Math.max(0.0001, layer.transform.scale.y) };
}
function findLayer(layers: readonly Layer[], layerId: string): Layer | undefined {

View File

@@ -1,10 +1,13 @@
import { commandIds } from "@commands/ids";
import type { Asset } from "@core/asset";
import type { LayerId } from "@core/id";
import type { InpaintRegionId } from "@core/id";
import type { AppStore } from "@editor/store";
import { analyzeMaskSource, applyMaskRasterOperation, createSolidMaskSource, type MaskAnalysis, type MaskRasterOperation } from "@platform/browser/maskRaster";
export type { MaskAnalysis, MaskRasterOperation };
export function analyzeMask(asset: Asset): Promise<MaskAnalysis> { return analyzeMaskSource(asset.source, asset.intrinsicSize.w, asset.intrinsicSize.h); }
export async function runMaskOperation(maskLayerId: LayerId, asset: Asset, operation: MaskRasterOperation, dispatch: AppStore["dispatch"]) { const source = await applyMaskRasterOperation(asset.source, asset.intrinsicSize.w, asset.intrinsicSize.h, operation); dispatch(commandIds.documentApplyLayerMaskOperation, { maskLayerId, source, mimeType: "image/png", operation }); }
export async function runInpaintRegionOperation(regionId: InpaintRegionId, asset: Asset, operation: MaskRasterOperation, dispatch: AppStore["dispatch"]) { const source = await applyMaskRasterOperation(asset.source, asset.intrinsicSize.w, asset.intrinsicSize.h, operation); dispatch(commandIds.documentApplyInpaintRegionMaskOperation, { regionId, source, mimeType: "image/png", operation }); }
export function createRefinementMask(width: number, height: number) { return createSolidMaskSource(width, height, "white"); }
export function createInpaintRegionMask(width: number, height: number) { return createSolidMaskSource(width, height, "black"); }

View File

@@ -0,0 +1,34 @@
import { commandIds } from "@commands/ids";
import type { Vec2D } from "@core/geometry";
import type { AppStore } from "@editor/store";
import { mergeMaskSources } from "@platform/browser/maskRaster";
import { requestSemanticSelection } from "@platform/comfy/generationClient";
import { runGenerationJob } from "@operations/generation/generationJob";
import { documentPointToAssetPoint, resolveMaskSelectionTarget } from "./magic-wand";
export async function applySemanticSelectionAt(store: AppStore, point: Vec2D, mode: "replace" | "add" | "subtract") {
const state = store.getState();
if (state.editor.tools.activeTool !== "semanticSelect") return false;
const target = resolveMaskSelectionTarget(state.document, state.editor);
if (!target?.inpaintRegion || !target.maskAsset) return true;
const region = target.inpaintRegion;
const maskAsset = target.maskAsset;
const assetPoint = documentPointToAssetPoint(point, target.layer, target.asset.intrinsicSize);
if (assetPoint.x < 0 || assetPoint.y < 0 || assetPoint.x >= target.asset.intrinsicSize.w || assetPoint.y >= target.asset.intrinsicSize.h) return true;
const controller = new AbortController();
await runGenerationJob({
kind: "mask",
label: "Selecting object",
dispatch: store.dispatch,
signal: controller.signal,
task: async (signal, report) => {
report(0.1, "Sending point to SAM3");
const result = await requestSemanticSelection({ inputImage: target.asset.source, x: assetPoint.x, y: assetPoint.y }, signal);
report(0.85, "Merging object mask");
const source = await mergeMaskSources(maskAsset.source, result.source, Math.round(target.asset.intrinsicSize.w), Math.round(target.asset.intrinsicSize.h), mode);
store.dispatch(commandIds.documentApplyInpaintRegionMaskOperation, { regionId: region.id, source, mimeType: "image/png", operation: { type: "magicWand" } });
report(1, "Object selected");
},
});
return true;
}

View File

@@ -1,12 +1,14 @@
import { commandIds } from "@commands/ids";
import type { ImageDocument } from "@core/document";
import type { Vec2D } from "@core/geometry";
import type { AssetId, LayerId } from "@core/id";
import type { Layer } from "@core/layer";
import { getLayerMask } from "@core/layer-mask-utils";
import type { RasterLayer } from "@core/raster-layer";
import type { MaskEditState, SelectionState } from "@editor/state";
import { isPanInteractionMode, type ToolState } from "@editor/tools";
import type { AppStore } from "@editor/store";
import { brushSurfaceDataUrl, brushSurfaceObjectUrl, cancelFrame, createBrushSurface, drawBrushSegment, releaseObjectUrl, scheduleFrame, type BrushSurface } from "@platform/browser/brushRaster";
import { brushSurfaceDataUrl, brushSurfaceObjectUrl, cancelFrame, createBrushSurface, drawBrushSegment, drawFeatherSegment, releaseObjectUrl, scheduleFrame, type BrushSurface } from "@platform/browser/brushRaster";
export type BrushSession = {
layerId: string;
@@ -16,7 +18,7 @@ export type BrushSession = {
surface: BrushSurface;
ready: Promise<boolean>;
previousPoint: Vec2D;
mode: "brush" | "eraser";
mode: "brush" | "eraser" | "feather";
changed?: boolean;
pending?: Promise<void>;
cancelled?: boolean;
@@ -25,6 +27,13 @@ export type BrushSession = {
previewInFlight?: boolean;
previewFrame?: number;
previewSource?: string;
targetLayer: RasterLayer;
featherRadius?: number;
pendingLayerMask?: {
targetLayerId: LayerId;
asset: ImageDocument["assets"][number];
maskLayer: RasterLayer;
};
};
export type BrushTargetEditorState = {
@@ -34,11 +43,9 @@ export type BrushTargetEditorState = {
};
export function beginBrushSession(document: ImageDocument, editor: BrushTargetEditorState, point: Vec2D): BrushSession | undefined {
const layer = resolveBrushTargetLayer(document, editor);
if (!layer || (editor.tools.activeTool !== "brush" && editor.tools.activeTool !== "eraser")) return undefined;
const asset = document.assets.find((candidate) => candidate.id === layer.assetId);
if (!asset) return undefined;
const target = resolveBrushTarget(document, editor);
if (!target || !isPaintTool(editor.tools.activeTool)) return undefined;
const { layer, asset } = target;
const surface = createBrushSurface(asset.intrinsicSize.w, asset.intrinsicSize.h, asset.source);
if (!surface) return undefined;
@@ -52,17 +59,20 @@ export function beginBrushSession(document: ImageDocument, editor: BrushTargetEd
ready: surface.ready,
previousPoint: point,
mode: editor.tools.activeTool,
targetLayer: layer,
pendingLayerMask: target.pendingLayerMask,
};
return session;
}
export function canPreviewBrush(document: ImageDocument, editor: BrushTargetEditorState): boolean {
return Boolean(resolveBrushTargetLayer(document, editor));
if (editor.tools.activeTool === "feather" && !editor.maskEdit) return Boolean(resolveSelectedFeatherLayer(document, editor));
return Boolean(resolveBrushTarget(document, editor));
}
export function brushUnavailableHint(document: ImageDocument, editor: BrushTargetEditorState): string | undefined {
if (isPanInteractionMode(editor.tools.interactionMode) || (editor.tools.activeTool !== "brush" && editor.tools.activeTool !== "eraser")) return undefined;
if (resolveBrushTargetLayer(document, editor)) return undefined;
if (isPanInteractionMode(editor.tools.interactionMode) || !isPaintTool(editor.tools.activeTool)) return undefined;
if (canPreviewBrush(document, editor)) return undefined;
const layerId = editor.maskEdit?.maskLayerId ?? editor.selection.layerIds[0];
if (!layerId) {
@@ -73,20 +83,73 @@ export function brushUnavailableHint(document: ImageDocument, editor: BrushTarge
const layer = findLayer(document.artboards.flatMap((artboard) => artboard.layers), layerId);
if (!layer) return "Select a raster layer or layer mask to paint.";
if (layer.locked) return "Unlock this layer before painting.";
if (layer.type === "image") return "Image layers are non-destructive. Add a layer mask to paint or erase.";
if (layer.type === "image" && editor.tools.activeTool !== "feather") return "Image layers are non-destructive. Add a layer mask to paint or erase.";
if (layer.type === "group") return "Select a raster layer inside the group to paint.";
if (!editor.maskEdit && !layer.visible) return "Show this layer before painting.";
return "Select a raster layer or layer mask to paint.";
}
function resolveBrushTargetLayer(document: ImageDocument, editor: BrushTargetEditorState): RasterLayer | undefined {
if (isPanInteractionMode(editor.tools.interactionMode) || (editor.tools.activeTool !== "brush" && editor.tools.activeTool !== "eraser")) return undefined;
function resolveBrushTarget(document: ImageDocument, editor: BrushTargetEditorState): { layer: RasterLayer; asset: ImageDocument["assets"][number]; pendingLayerMask?: BrushSession["pendingLayerMask"] } | undefined {
if (isPanInteractionMode(editor.tools.interactionMode) || !isPaintTool(editor.tools.activeTool)) return undefined;
if (editor.maskEdit?.kind === "inpaintRegion") {
const target = findLayer(document.artboards.flatMap((artboard) => artboard.layers), editor.maskEdit.targetLayerId);
const asset = document.assets.find((candidate) => candidate.id === editor.maskEdit?.maskAssetId);
if (!target || (target.type !== "image" && target.type !== "raster") || target.locked || !asset) return undefined;
return {
layer: { ...target, type: "raster", assetId: asset.id },
asset,
};
}
const editingMask = Boolean(editor.maskEdit);
const layerId = editor.maskEdit?.maskLayerId ?? editor.selection.layerIds[0];
if (!layerId) return undefined;
const layer = findRasterLayer(document.artboards.flatMap((artboard) => artboard.layers), layerId);
if (!layer || layer.locked || (!editingMask && !layer.visible)) return undefined;
return layer;
const layers = document.artboards.flatMap((artboard) => artboard.layers);
const selectedLayer = findLayer(layers, layerId);
if (!selectedLayer || selectedLayer.locked || (!editingMask && !selectedLayer.visible)) return undefined;
if (editor.tools.activeTool === "feather" && !editingMask && (selectedLayer.type === "image" || selectedLayer.type === "raster")) {
const maskLayerId = getLayerMask(selectedLayer)?.maskLayerId;
const attachedMask = maskLayerId ? findLayer(layers, maskLayerId) : undefined;
const attachedMaskLayer = attachedMask && (attachedMask.type === "image" || attachedMask.type === "raster") ? { ...attachedMask, type: "raster" as const } : undefined;
const maskAsset = attachedMaskLayer ? document.assets.find((candidate) => candidate.id === attachedMaskLayer.assetId) : undefined;
if (attachedMaskLayer && maskAsset) return { layer: attachedMaskLayer, asset: maskAsset };
const sourceAsset = document.assets.find((candidate) => candidate.id === selectedLayer.assetId);
if (!sourceAsset) return undefined;
const width = Math.max(1, Math.round(sourceAsset.intrinsicSize.w));
const height = Math.max(1, Math.round(sourceAsset.intrinsicSize.h));
const source = opaqueMaskSource(width, height);
const assetId = crypto.randomUUID() as AssetId;
const newMaskLayerId = crypto.randomUUID() as LayerId;
const sourceRect = selectedLayer.sourceRect ?? { x: 0, y: 0, w: width, h: height };
const maskLayer: RasterLayer = {
id: newMaskLayerId,
type: "raster",
name: `${selectedLayer.name} Mask`,
visible: true,
locked: false,
opacity: 1,
assetId,
transform: {
position: {
x: selectedLayer.transform.position.x + sourceRect.x * selectedLayer.transform.scale.x,
y: selectedLayer.transform.position.y + sourceRect.y * selectedLayer.transform.scale.y,
},
scale: {
x: (sourceRect.w * selectedLayer.transform.scale.x) / width,
y: (sourceRect.h * selectedLayer.transform.scale.y) / height,
},
rotation: selectedLayer.transform.rotation,
},
};
const asset = { id: assetId, name: maskLayer.name, mimeType: "image/svg+xml", source, intrinsicSize: { w: width, h: height } };
return { layer: maskLayer, asset, pendingLayerMask: { targetLayerId: selectedLayer.id, asset, maskLayer } };
}
const layer = selectedLayer.type === "raster" ? selectedLayer : undefined;
if (!layer) return undefined;
const asset = document.assets.find((candidate) => candidate.id === layer.assetId);
return asset ? { layer, asset } : undefined;
}
export function updateBrushSession(options: {
@@ -96,27 +159,53 @@ export function updateBrushSession(options: {
color: string;
size: number;
hardness: number;
opacity: number;
flow: number;
smoothing: number;
pressure: number;
pressureSize: boolean;
featherRadius?: number;
featherStrength?: number;
}): BrushSession {
const state = options.store.getState();
const layer = findRasterLayer(state.document.artboards.flatMap((artboard) => artboard.layers), options.session.layerId);
if (!layer || layer.assetId !== options.session.assetId) return options.session;
const asset = state.document.assets.find((candidate) => candidate.id === options.session.assetId);
if (!asset && !options.session.pendingLayerMask) return options.session;
const layer = options.session.targetLayer;
const from = options.session.previousPoint;
const to = options.point;
const smoothing = Math.max(0, Math.min(100, options.smoothing)) / 100;
const follow = 1 - smoothing * 0.85;
const to = { x: from.x + (options.point.x - from.x) * follow, y: from.y + (options.point.y - from.y) * follow };
options.session.previousPoint = to;
options.session.pending = (options.session.pending ?? Promise.resolve())
.then(async () => {
if (options.session.cancelled) return;
if (!(await options.session.ready) || options.session.cancelled) return;
const assetFrom = documentPointToAssetPoint(from, layer, options.session.width, options.session.height);
const assetTo = documentPointToAssetPoint(to, layer, options.session.width, options.session.height);
const size = options.size * (options.pressureSize ? Math.max(0.1, options.pressure) : 1);
if (options.session.mode === "feather") {
options.session.featherRadius = options.featherRadius ?? 16;
drawFeatherSegment(options.session.surface, {
from: assetFrom,
to: assetTo,
size,
radius: options.featherRadius ?? 16,
strength: options.featherStrength ?? 65,
});
} else {
drawBrushSegment(options.session.surface, {
from: documentPointToAssetPoint(from, layer, options.session.width, options.session.height),
to: documentPointToAssetPoint(to, layer, options.session.width, options.session.height),
from: assetFrom,
to: assetTo,
color: state.editor.maskEdit ? "#ffffff" : options.color,
size: options.size,
size,
hardness: options.hardness,
opacity: options.opacity,
flow: options.flow,
mode: options.session.mode,
});
}
if (options.session.cancelled) return;
options.session.changed = true;
@@ -135,8 +224,22 @@ export async function commitBrushSession(options: { store: AppStore; session: Br
if (source) {
const state = options.store.getState();
const maskEdit = state.editor.maskEdit;
if (maskEdit?.maskLayerId === options.session.layerId) {
options.store.dispatch(commandIds.documentApplyLayerMaskOperation, { maskLayerId: maskEdit.maskLayerId, source, mimeType: "image/png", operation: { type: "paint" } });
const operation = options.session.mode === "feather"
? { type: "feather" as const, radius: Math.round(options.session.featherRadius ?? 16) }
: { type: "paint" as const };
if (options.session.pendingLayerMask) {
options.store.dispatch(commandIds.documentAddLayerMask, {
layerId: options.session.pendingLayerMask.targetLayerId,
asset: { ...options.session.pendingLayerMask.asset, source, mimeType: "image/png" },
maskLayer: options.session.pendingLayerMask.maskLayer,
activeTool: "feather",
});
} else if (maskEdit?.kind === "inpaintRegion" && maskEdit.inpaintRegionId && maskEdit.maskAssetId === options.session.assetId) {
options.store.dispatch(commandIds.documentApplyInpaintRegionMaskOperation, { regionId: maskEdit.inpaintRegionId, source, mimeType: "image/png", operation });
} else if (maskEdit?.maskLayerId === options.session.layerId) {
options.store.dispatch(commandIds.documentApplyLayerMaskOperation, { maskLayerId: maskEdit.maskLayerId, source, mimeType: "image/png", operation });
} else if (isAttachedLayerMask(state.document, options.session.layerId)) {
options.store.dispatch(commandIds.documentApplyLayerMaskOperation, { maskLayerId: options.session.layerId, source, mimeType: "image/png", operation });
} else {
options.store.dispatch(commandIds.documentUpdateAssetSource, { assetId: options.session.assetId, source });
}
@@ -145,6 +248,33 @@ export async function commitBrushSession(options: { store: AppStore; session: Br
closeBrushStrokePreview(options.session);
}
function isPaintTool(tool: ToolState["activeTool"]): tool is "brush" | "eraser" | "feather" {
return tool === "brush" || tool === "eraser" || tool === "feather";
}
function resolveSelectedFeatherLayer(document: ImageDocument, editor: BrushTargetEditorState) {
if (isPanInteractionMode(editor.tools.interactionMode)) return undefined;
const layerId = editor.selection.layerIds[0];
if (!layerId) return undefined;
const layer = findLayer(document.artboards.flatMap((artboard) => artboard.layers), layerId);
return layer && (layer.type === "image" || layer.type === "raster") && layer.visible && !layer.locked ? layer : undefined;
}
function isAttachedLayerMask(document: ImageDocument, maskLayerId: string) {
const stack = document.artboards.flatMap((artboard) => artboard.layers);
while (stack.length > 0) {
const layer = stack.pop();
if (!layer) continue;
if (getLayerMask(layer)?.maskLayerId === maskLayerId) return true;
if (layer.type === "group") stack.push(...layer.children);
}
return false;
}
function opaqueMaskSource(width: number, height: number) {
return `data:image/svg+xml,${encodeURIComponent(`<svg xmlns="http://www.w3.org/2000/svg" width="${width}" height="${height}" viewBox="0 0 ${width} ${height}"><rect width="${width}" height="${height}" fill="white"/></svg>`)}`;
}
export function cancelBrushSession(options: { store: AppStore; session: BrushSession }) {
options.session.cancelled = true;
options.store.dispatch(commandIds.toolSetBrushStrokePreview, undefined);
@@ -152,9 +282,22 @@ export function cancelBrushSession(options: { store: AppStore; session: BrushSes
}
function documentPointToAssetPoint(point: Vec2D, layer: RasterLayer, width: number, height: number): Vec2D {
const source = layer.sourceRect ?? { x: 0, y: 0, w: width, h: height };
const destination = {
x: layer.transform.position.x + source.x * layer.transform.scale.x,
y: layer.transform.position.y + source.y * layer.transform.scale.y,
w: source.w * layer.transform.scale.x,
h: source.h * layer.transform.scale.y,
};
const center = { x: destination.x + destination.w / 2, y: destination.y + destination.h / 2 };
const cos = Math.cos(-layer.transform.rotation);
const sin = Math.sin(-layer.transform.rotation);
const dx = point.x - center.x;
const dy = point.y - center.y;
const unrotated = { x: center.x + dx * cos - dy * sin, y: center.y + dx * sin + dy * cos };
return {
x: ((point.x - layer.transform.position.x) / Math.max(0.0001, layer.transform.scale.x) / width) * width,
y: ((point.y - layer.transform.position.y) / Math.max(0.0001, layer.transform.scale.y) / height) * height,
x: source.x + (unrotated.x - destination.x) / Math.max(0.0001, layer.transform.scale.x),
y: source.y + (unrotated.y - destination.y) / Math.max(0.0001, layer.transform.scale.y),
};
}
@@ -190,7 +333,13 @@ async function publishBrushStrokePreview(options: { store: AppStore; session: Br
const previousSource = options.session.previewSource;
options.session.previewSource = source;
options.store.dispatch(commandIds.toolSetBrushStrokePreview, { layerId: options.session.layerId, assetId: options.session.assetId, source });
options.store.dispatch(commandIds.toolSetBrushStrokePreview, {
layerId: options.session.layerId,
assetId: options.session.assetId,
source,
pendingTargetLayerId: options.session.pendingLayerMask?.targetLayerId,
intrinsicSize: options.session.pendingLayerMask ? { w: options.session.width, h: options.session.height } : undefined,
});
releaseObjectUrl(previousSource);
if (options.session.previewRequested) requestBrushStrokePreview(options);
@@ -208,11 +357,6 @@ function closeBrushStrokePreview(session: BrushSession) {
}
}
function findRasterLayer(layers: Layer[], layerId: string): RasterLayer | undefined {
const layer = findLayer(layers, layerId);
return layer?.type === "raster" ? layer : undefined;
}
function findLayer(layers: Layer[], layerId: string): Layer | undefined {
for (const layer of layers) {
if (layer.id === layerId) return layer;

View File

@@ -0,0 +1,90 @@
import { describe, expect, test } from "bun:test";
import type { ImageDocument } from "@core/document";
import { CURRENT_PROJECT_VERSION, parseProject, projectFileName, serializeProject } from "./format";
describe("project format", () => {
test("round-trips a versioned project with embedded assets", () => {
const document = projectDocument();
const result = parseProject(serializeProject(document, "2026-07-10T12:00:00.000Z"));
expect(result.version).toBe(CURRENT_PROJECT_VERSION);
expect(result.savedAt).toBe("2026-07-10T12:00:00.000Z");
expect(result.document).toEqual(document);
});
test("migrates a legacy bare document", () => {
const result = parseProject(JSON.stringify(projectDocument()));
expect(result.version).toBe(CURRENT_PROJECT_VERSION);
expect(result.savedAt).toBe("1970-01-01T00:00:00.000Z");
});
test("rejects unknown versions and invalid JSON", () => {
expect(() => parseProject("not json")).toThrow("not valid JSON");
expect(() => parseProject(JSON.stringify({ format: "image-studio-project", version: 99, savedAt: "now", document: projectDocument() }))).toThrow("Unsupported project version");
});
test("enforces project-owned embedded assets and valid references", () => {
const externalAsset = projectDocument();
externalAsset.assets[0]!.source = "blob:https://example.test/asset";
expect(() => serializeProject(externalAsset)).toThrow("not embedded");
const missingAsset = projectDocument();
missingAsset.assets = [];
expect(() => serializeProject(missingAsset)).toThrow("references missing asset");
});
test("rejects invalid persisted layer crops", () => {
const document = projectDocument();
const layer = document.artboards[0]!.layers[0]!;
if (layer.type !== "image" && layer.type !== "raster") throw new Error("Expected image layer");
layer.sourceRect = { x: 0, y: 0, w: 0, h: 10 };
const source = JSON.stringify({ format: "image-studio-project", version: 1, savedAt: "2026-07-10T12:00:00.000Z", document });
expect(() => parseProject(source)).toThrow("invalid source crop");
});
test("round-trips adjustment layers and rejects out-of-range settings", () => {
const document = projectDocument();
document.artboards[0]!.layers.unshift({ id: "adjust", type: "adjustment", name: "Color", visible: true, locked: false, opacity: 1, transform: { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 }, adjustment: { brightness: .2, contrast: 0, saturation: 0, colorBalance: { red: 0, green: 0, blue: 0 } } });
expect(parseProject(serializeProject(document)).document.artboards[0]?.layers[0]?.type).toBe("adjustment");
const invalid = JSON.parse(serializeProject(document)); invalid.document.artboards[0].layers[0].adjustment.brightness = 4;
expect(() => parseProject(JSON.stringify(invalid))).toThrow("invalid settings");
});
test("rejects nested adjustment scope", () => {
const document = projectDocument();
document.artboards[0]!.layers = [{ id: "group", type: "group", name: "Group", visible: true, locked: false, opacity: 1, transform: { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 }, children: [{ id: "adjust", type: "adjustment", name: "Color", visible: true, locked: false, opacity: 1, transform: { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 }, adjustment: { brightness: 0, contrast: 0, saturation: 0, colorBalance: { red: 0, green: 0, blue: 0 } } }] }];
expect(() => parseProject(serializeProject(document))).toThrow("artboard level");
});
test("round-trips valid text and rejects unsupported typography", () => {
const document = projectDocument();
document.artboards[0]!.layers.unshift({ id: "text", type: "text", name: "Text", visible: true, locked: false, opacity: .8, transform: { position: { x: 1, y: 2 }, scale: { x: 1, y: 1 }, rotation: .2 }, content: "Hello", style: { fontFamily: "Arial", fontSize: 20, fontWeight: 700, fontStyle: "italic", color: "#123456", alignment: "right", lineHeight: 1.2 } });
expect(parseProject(serializeProject(document)).document.artboards[0]?.layers[0]?.type).toBe("text");
const invalid = JSON.parse(serializeProject(document)); invalid.document.artboards[0].layers[0].style.fontFamily = "Web Font";
expect(() => parseProject(JSON.stringify(invalid))).toThrow("invalid typography");
});
test("creates safe project file names", () => {
expect(projectFileName(" Summer / Study ")).toBe("Summer-Study.image-studio.json");
expect(projectFileName("***")).toBe("untitled.image-studio.json");
});
});
function projectDocument(): ImageDocument {
return {
id: "document-1",
name: "Test Project",
version: 1,
assets: [{ id: "asset-1", name: "pixels.png", mimeType: "image/png", source: "data:image/png;base64,AA==", intrinsicSize: { w: 10, h: 20 } }],
inpaintRegions: [],
artboards: [{
id: "artboard-1",
name: "Board",
bounds: { x: 0, y: 0, w: 100, h: 100 },
backgroundColor: "transparent",
visible: true,
locked: false,
layers: [{ id: "layer-1", type: "image", name: "Pixels", visible: true, locked: false, opacity: 1, assetId: "asset-1", sourceRect: { x: 2, y: 3, w: 6, h: 12 }, transform: { position: { x: 0, y: 0 }, scale: { x: 1, y: 1 }, rotation: 0 } }],
}],
};
}

View File

@@ -0,0 +1,184 @@
import type { ImageDocument } from "@core/document";
import type { Layer } from "@core/layer";
import type { Rect } from "@core/geometry";
import { isValidTextStyle, type TextStyle } from "@core/text-layer";
export const PROJECT_FORMAT = "image-studio-project";
export const CURRENT_PROJECT_VERSION = 2;
export type ProjectFile = {
format: typeof PROJECT_FORMAT;
version: typeof CURRENT_PROJECT_VERSION;
savedAt: string;
document: ImageDocument;
};
export function serializeProject(document: ImageDocument, savedAt = new Date().toISOString()): string {
assertDocumentAssetOwnership(document);
return JSON.stringify({ format: PROJECT_FORMAT, version: CURRENT_PROJECT_VERSION, savedAt, document } satisfies ProjectFile);
}
export function parseProject(source: string): ProjectFile {
let value: unknown;
try {
value = JSON.parse(source);
} catch {
throw new Error("The selected file is not valid JSON.");
}
const migrated = migrateProject(value);
assertImageDocument(migrated.document);
assertDocumentAssetOwnership(migrated.document);
return migrated;
}
export function projectFileName(documentName: string): string {
const base = documentName.trim().replace(/[^a-z0-9._-]+/gi, "-").replace(/^-+|-+$/g, "") || "untitled";
return `${base}.image-studio.json`;
}
function migrateProject(value: unknown): ProjectFile {
if (!isRecord(value)) throw new Error("The project file must contain an object.");
if (value.format === PROJECT_FORMAT) {
if (value.version === 1 && isRecord(value.document)) {
return {
format: PROJECT_FORMAT,
version: CURRENT_PROJECT_VERSION,
savedAt: typeof value.savedAt === "string" ? value.savedAt : new Date(0).toISOString(),
document: { ...value.document, inpaintRegions: [] } as unknown as ImageDocument,
};
}
if (value.version !== CURRENT_PROJECT_VERSION) {
throw new Error(`Unsupported project version: ${String(value.version)}.`);
}
if (typeof value.savedAt !== "string") throw new Error("The project is missing its save timestamp.");
return value as ProjectFile;
}
// Legacy exports stored ImageDocument directly. Loading upgrades them to the current format.
if (looksLikeDocument(value)) {
return {
format: PROJECT_FORMAT,
version: CURRENT_PROJECT_VERSION,
savedAt: new Date(0).toISOString(),
document: { ...value, inpaintRegions: Array.isArray(value.inpaintRegions) ? value.inpaintRegions : [] } as ImageDocument,
};
}
throw new Error("This is not an Image Studio project file.");
}
function assertImageDocument(value: unknown): asserts value is ImageDocument {
if (!isRecord(value) || typeof value.id !== "string" || typeof value.name !== "string" || typeof value.version !== "number") {
throw new Error("The project contains an invalid document.");
}
if (!Array.isArray(value.artboards) || !Array.isArray(value.assets) || !Array.isArray(value.inpaintRegions)) throw new Error("The project document is incomplete.");
for (const asset of value.assets) {
if (!isRecord(asset) || typeof asset.id !== "string" || typeof asset.name !== "string" || typeof asset.mimeType !== "string" || typeof asset.source !== "string" || !isSize(asset.intrinsicSize)) {
throw new Error("The project contains an invalid asset.");
}
}
for (const artboard of value.artboards) {
if (!isRecord(artboard) || typeof artboard.id !== "string" || typeof artboard.name !== "string" || !isRect(artboard.bounds) || !Array.isArray(artboard.layers)) {
throw new Error("The project contains an invalid artboard.");
}
assertLayers(artboard.layers, false);
}
for (const region of value.inpaintRegions) {
if (!isRecord(region) || typeof region.id !== "string" || typeof region.name !== "string" || typeof region.targetLayerId !== "string" || typeof region.maskAssetId !== "string" || typeof region.enabled !== "boolean") {
throw new Error("The project contains an invalid inpaint region.");
}
}
}
export function assertDocumentAssetOwnership(document: ImageDocument): void {
const assetIds = new Set<string>();
for (const asset of document.assets) {
if (assetIds.has(asset.id)) throw new Error(`Duplicate asset id: ${asset.id}.`);
assetIds.add(asset.id);
if (!isOwnedAssetSource(asset.source)) throw new Error(`Asset ${asset.name} is not embedded in the project.`);
}
const layerIds = new Set<string>();
for (const artboard of document.artboards) {
walkLayers(artboard.layers, (layer) => {
if (layerIds.has(layer.id)) throw new Error(`Duplicate layer id: ${layer.id}.`);
layerIds.add(layer.id);
if ((layer.type === "image" || layer.type === "raster") && !assetIds.has(layer.assetId)) {
throw new Error(`Layer ${layer.name} references missing asset ${layer.assetId}.`);
}
});
}
for (const region of document.inpaintRegions) {
if (!assetIds.has(region.maskAssetId)) throw new Error(`Inpaint region ${region.name} references missing mask asset ${region.maskAssetId}.`);
if (!layerIds.has(region.targetLayerId)) throw new Error(`Inpaint region ${region.name} references missing target layer ${region.targetLayerId}.`);
}
}
function isOwnedAssetSource(source: string): boolean {
return source.startsWith("data:");
}
function assertLayers(value: unknown[], nested: boolean): asserts value is Layer[] {
for (const layer of value) {
if (!isRecord(layer) || typeof layer.id !== "string" || typeof layer.name !== "string" || typeof layer.type !== "string" || typeof layer.visible !== "boolean" || typeof layer.locked !== "boolean" || typeof layer.opacity !== "number" || !isTransform(layer.transform)) {
throw new Error("The project contains an invalid layer.");
}
if (layer.type === "group") {
if (!Array.isArray(layer.children)) throw new Error("A project group is missing its children.");
assertLayers(layer.children, true);
} else if (layer.type === "adjustment") {
if (nested) throw new Error("Adjustment layers must stay at artboard level.");
if (!isAdjustment(layer.adjustment)) throw new Error("A project adjustment layer contains invalid settings.");
if (layer.layerMask !== undefined || layer.clippingMask !== undefined) throw new Error("Adjustment layers do not support masks.");
} else if (layer.type === "text") {
if (typeof layer.content !== "string" || !layer.content.trim() || !isRecord(layer.style) || !isValidTextStyle(layer.style as TextStyle)) throw new Error("A project text layer contains invalid typography.");
if (layer.layerMask !== undefined || layer.clippingMask !== undefined) throw new Error("Text layers do not support masks. Rasterize text before masking.");
} else if ((layer.type === "image" || layer.type === "raster") && typeof layer.assetId !== "string") {
throw new Error("A project layer is missing its asset reference.");
} else if ((layer.type === "image" || layer.type === "raster") && layer.sourceRect !== undefined && (!isRect(layer.sourceRect) || layer.sourceRect.w < 1 || layer.sourceRect.h < 1)) {
throw new Error("A project layer contains an invalid source crop.");
} else if (layer.type !== "image" && layer.type !== "raster" && layer.type !== "text") {
throw new Error(`Unsupported layer type: ${layer.type}.`);
}
}
}
function isAdjustment(value: unknown): boolean {
if (!isRecord(value) || !isRecord(value.colorBalance)) return false;
return [value.brightness, value.contrast, value.saturation, value.colorBalance.red, value.colorBalance.green, value.colorBalance.blue]
.every((number) => isFiniteNumber(number) && number >= -1 && number <= 1);
}
function walkLayers(layers: Layer[], visit: (layer: Layer) => void): void {
for (const layer of layers) {
visit(layer);
if (layer.type === "group") walkLayers(layer.children, visit);
}
}
function looksLikeDocument(value: Record<string, unknown>): boolean {
return typeof value.id === "string" && Array.isArray(value.artboards) && Array.isArray(value.assets);
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
function isSize(value: unknown): boolean {
return isRecord(value) && isFiniteNumber(value.w) && isFiniteNumber(value.h) && value.w >= 0 && value.h >= 0;
}
function isRect(value: unknown): value is Rect {
return isRecord(value) && isFiniteNumber(value.x) && isFiniteNumber(value.y) && isFiniteNumber(value.w) && isFiniteNumber(value.h);
}
function isTransform(value: unknown): boolean {
return isRecord(value) && isRecord(value.position) && isFiniteNumber(value.position.x) && isFiniteNumber(value.position.y) && isRecord(value.scale) && isFiniteNumber(value.scale.x) && isFiniteNumber(value.scale.y) && isFiniteNumber(value.rotation);
}
function isFiniteNumber(value: unknown): value is number {
return typeof value === "number" && Number.isFinite(value);
}

View File

@@ -0,0 +1,99 @@
import { describe, expect, test } from "bun:test";
import { commandIds } from "@commands/ids";
import { documentAddArtboardCommand, documentRenameArtboardCommand } from "@commands/document";
import { projectOpenCommand } from "@commands/project";
import { createCommandRegistry } from "@commands/registry";
import { createInitialAppState } from "@editor/initial-state";
import { createAppStore } from "@editor/store";
import type { RecoveryStorage } from "@platform/projectStorage";
import { createProjectLifecycle, RECOVERY_STORAGE_KEY } from "./lifecycle";
import { parseProject, serializeProject } from "./format";
describe("project lifecycle", () => {
test("autosaves document changes after the debounce", async () => {
const app = createTestApp();
const storage = memoryStorage();
let scheduled: (() => void) | undefined;
const lifecycle = createProjectLifecycle({
store: app.store,
storage,
schedule(callback) {
scheduled = callback;
return 1 as unknown as ReturnType<typeof setTimeout>;
},
cancel() {},
});
const artboardId = app.store.getState().document.artboards[0]!.id;
app.store.dispatch(commandIds.documentRenameArtboard, { id: artboardId, name: "Recovered board" });
expect(await storage.read(RECOVERY_STORAGE_KEY)).toBeNull();
scheduled?.();
await Promise.resolve();
expect(parseProject((await storage.read(RECOVERY_STORAGE_KEY))!).document.artboards[0]!.name).toBe("Recovered board");
lifecycle.dispose();
});
test("recovers a document through the project command and clears history", async () => {
const app = createTestApp();
const recovered = createTestApp("Recovered").store.getState().document;
const storage = memoryStorage([[RECOVERY_STORAGE_KEY, serializeProject(recovered)]]);
const lifecycle = createProjectLifecycle({ store: app.store, storage });
expect((await lifecycle.recover())?.document.name).toBe("Recovered");
expect(app.store.getState().document.name).toBe("Recovered");
expect(app.store.getState().history).toEqual({ past: [], future: [] });
lifecycle.dispose();
});
test("discards corrupted recovery and clears recovery after explicit save", async () => {
const app = createTestApp();
const storage = memoryStorage([[RECOVERY_STORAGE_KEY, "broken"]]);
const lifecycle = createProjectLifecycle({ store: app.store, storage });
expect(await lifecycle.recover()).toBeUndefined();
expect(await storage.read(RECOVERY_STORAGE_KEY)).toBeNull();
await storage.write(RECOVERY_STORAGE_KEY, lifecycle.snapshot());
await lifecycle.markSaved();
expect(await storage.read(RECOVERY_STORAGE_KEY)).toBeNull();
lifecycle.dispose();
});
test("explicit save cancels a pending recovery write", async () => {
const app = createTestApp();
const storage = memoryStorage();
let scheduled: (() => void) | undefined;
let cancelled = false;
const lifecycle = createProjectLifecycle({
store: app.store,
storage,
schedule(callback) {
scheduled = callback;
return 1;
},
cancel() { cancelled = true; },
});
const artboardId = app.store.getState().document.artboards[0]!.id;
app.store.dispatch(commandIds.documentRenameArtboard, { id: artboardId, name: "Saved" });
await lifecycle.markSaved();
expect(cancelled).toBe(true);
expect(await storage.read(RECOVERY_STORAGE_KEY)).toBeNull();
expect(scheduled).toBeDefined();
lifecycle.dispose();
});
});
function memoryStorage(entries: [string, string][] = []): RecoveryStorage {
const values = new Map(entries);
return {
read: async (key) => values.get(key) ?? null,
write: async (key, value) => { values.set(key, value); },
remove: async (key) => { values.delete(key); },
};
}
function createTestApp(name = "Test") {
const store = createAppStore(createInitialAppState(name), createCommandRegistry([projectOpenCommand, documentAddArtboardCommand, documentRenameArtboardCommand]));
store.dispatch(commandIds.documentAddArtboard, { id: `${name}-artboard`, name: "Board", bounds: { x: 0, y: 0, w: 100, h: 100 } });
return { store };
}

View File

@@ -0,0 +1,86 @@
import { commandIds } from "@commands/ids";
import type { AppStore } from "@editor/store";
import type { ProjectFile } from "./format";
import { parseProject, serializeProject } from "./format";
import type { RecoveryStorage } from "@platform/projectStorage";
export const RECOVERY_STORAGE_KEY = "image-studio.recovery.v1";
type TimerHandle = number | ReturnType<typeof setTimeout>;
export type ProjectLifecycle = {
recover(): Promise<ProjectFile | undefined>;
open(source: string): Promise<ProjectFile>;
snapshot(): string;
markSaved(): Promise<void>;
dispose(): void;
};
export function createProjectLifecycle(options: {
store: AppStore;
storage: RecoveryStorage;
debounceMs?: number;
schedule?: (callback: () => void, delay: number) => TimerHandle;
cancel?: (handle: TimerHandle) => void;
}): ProjectLifecycle {
const debounceMs = options.debounceMs ?? 750;
const schedule = options.schedule ?? setTimeout;
const cancel = options.cancel ?? ((handle: TimerHandle) => clearTimeout(handle));
let observedDocument = options.store.getState().document;
let pending: TimerHandle | undefined;
let disposed = false;
const cancelPending = () => {
if (pending === undefined) return;
cancel(pending);
pending = undefined;
};
const unsubscribe = options.store.subscribe((state) => {
if (state.document === observedDocument) return;
observedDocument = state.document;
cancelPending();
pending = schedule(() => {
pending = undefined;
void options.storage.write(RECOVERY_STORAGE_KEY, serializeProject(options.store.getState().document)).catch(() => undefined);
}, debounceMs);
});
const replace = (project: ProjectFile) => {
options.store.dispatch(commandIds.projectOpen, { document: project.document });
cancelPending();
observedDocument = options.store.getState().document;
return project;
};
return {
async recover() {
const source = await options.storage.read(RECOVERY_STORAGE_KEY);
if (!source || disposed) return undefined;
try {
return replace(parseProject(source));
} catch {
await options.storage.remove(RECOVERY_STORAGE_KEY);
return undefined;
}
},
async open(source) {
const project = replace(parseProject(source));
await options.storage.remove(RECOVERY_STORAGE_KEY);
return project;
},
snapshot() {
return serializeProject(options.store.getState().document);
},
async markSaved() {
cancelPending();
observedDocument = options.store.getState().document;
await options.storage.remove(RECOVERY_STORAGE_KEY);
},
dispose() {
disposed = true;
unsubscribe();
cancelPending();
},
};
}

View File

@@ -1,10 +1,11 @@
{
"name": "bun-react-template",
"name": "image-studio",
"version": "0.1.0",
"private": true,
"type": "module",
"scripts": {
"dev": "bun --hot index.ts",
"dev:full": "./scripts/dev-with-comfy.sh",
"start": "NODE_ENV=production bun index.ts",
"build": "bun run build.ts",
"lint": "eslint .",

View File

@@ -0,0 +1,10 @@
import { describe, expect, test } from "bun:test";
import { adjustPixel } from "./exportArtboardPng";
import { neutralColorAdjustment } from "@core/adjustment-layer";
describe("adjustment pixel calculation", () => {
test("neutral adjustment is stable", () => expect(adjustPixel([20, 100, 220], neutralColorAdjustment)).toEqual([20, 100, 220]));
test("brightness, contrast, saturation and balance use deterministic ordering", () => {
expect(adjustPixel([100, 120, 140], { brightness: .1, contrast: .2, saturation: -.5, colorBalance: { red: .1, green: 0, blue: -.1 } })).toEqual([153, 150, 146]);
});
});

View File

@@ -0,0 +1,40 @@
import { describe, expect, test } from "bun:test";
import { blendFeatherPatch } from "./brushRaster";
describe("feather brush raster blending", () => {
test("mixes blurred mask values inside the stroke and preserves pixels outside it", () => {
const original = rgbaRow([255, 255, 255, 255, 255]);
const blurred = rgbaRow([0, 64, 128, 192, 255]);
blendFeatherPatch(original, blurred, 5, 1, { x: 0, y: 0 }, {
from: { x: 2.5, y: 0.5 },
to: { x: 2.5, y: 0.5 },
size: 3,
strength: 100,
});
expect(alphaValues(original)).toEqual([255, 64, 128, 192, 255]);
});
test("applies strength as a non-destructive mix", () => {
const original = rgbaRow([255]);
const blurred = rgbaRow([0]);
blendFeatherPatch(original, blurred, 1, 1, { x: 0, y: 0 }, {
from: { x: 0.5, y: 0.5 },
to: { x: 0.5, y: 0.5 },
size: 10,
strength: 50,
});
expect(alphaValues(original)).toEqual([128]);
});
});
function rgbaRow(alpha: number[]) {
return new Uint8ClampedArray(alpha.flatMap((value) => [255, 255, 255, value]));
}
function alphaValues(data: Uint8ClampedArray) {
return Array.from({ length: data.length / 4 }, (_, index) => data[index * 4 + 3]);
}

View File

@@ -5,19 +5,101 @@ export type BrushSurface = {
readonly resource: object;
};
type InternalSurface = BrushSurface & { canvas: HTMLCanvasElement; context: CanvasRenderingContext2D };
type InternalSurface = BrushSurface & {
canvas: HTMLCanvasElement;
context: CanvasRenderingContext2D;
featherSource: HTMLCanvasElement;
featherBlur: HTMLCanvasElement;
};
export function createBrushSurface(width: number, height: number, source: string): BrushSurface | undefined {
const canvas = document.createElement("canvas"); canvas.width = Math.max(1, Math.round(width)); canvas.height = Math.max(1, Math.round(height));
const context = canvas.getContext("2d"); if (!context) return undefined;
const surface = { width: canvas.width, height: canvas.height, canvas, context, resource: {}, ready: Promise.resolve(false) } as InternalSurface;
const surface = {
width: canvas.width,
height: canvas.height,
canvas,
context,
featherSource: document.createElement("canvas"),
featherBlur: document.createElement("canvas"),
resource: {},
ready: Promise.resolve(false),
} as InternalSurface;
(surface as { ready: Promise<boolean> }).ready = loadImage(source).then((image) => { context.clearRect(0, 0, canvas.width, canvas.height); context.drawImage(image, 0, 0, canvas.width, canvas.height); return true; }).catch(() => false);
return surface;
}
export function drawBrushSegment(surface: BrushSurface, options: { from: { x: number; y: number }; to: { x: number; y: number }; color: string; size: number; hardness: number; mode: "brush" | "eraser" }) {
export function drawBrushSegment(surface: BrushSurface, options: { from: { x: number; y: number }; to: { x: number; y: number }; color: string; size: number; hardness: number; opacity: number; flow: number; mode: "brush" | "eraser" }) {
const context = internal(surface).context; const hardness = Math.max(0, Math.min(100, options.hardness)) / 100;
context.save(); context.globalCompositeOperation = options.mode === "eraser" ? "destination-out" : "source-over"; context.strokeStyle = options.color; context.shadowColor = options.mode === "eraser" ? "rgba(0,0,0,1)" : options.color; context.shadowBlur = (1 - hardness) * options.size; context.lineWidth = options.size; context.lineCap = "round"; context.lineJoin = "round"; context.beginPath(); context.moveTo(options.from.x, options.from.y); context.lineTo(options.to.x, options.to.y); context.stroke(); context.restore();
context.save(); context.globalAlpha = Math.max(0, Math.min(1, options.opacity / 100)) * Math.max(0.01, Math.min(1, options.flow / 100)); context.globalCompositeOperation = options.mode === "eraser" ? "destination-out" : "source-over"; context.strokeStyle = options.color; context.shadowColor = options.mode === "eraser" ? "rgba(0,0,0,1)" : options.color; context.shadowBlur = (1 - hardness) * options.size; context.lineWidth = options.size; context.lineCap = "round"; context.lineJoin = "round"; context.beginPath(); context.moveTo(options.from.x, options.from.y); context.lineTo(options.to.x, options.to.y); context.stroke(); context.restore();
}
export function drawFeatherSegment(surface: BrushSurface, options: {
from: { x: number; y: number };
to: { x: number; y: number };
size: number;
radius: number;
strength: number;
}) {
const target = internal(surface);
const brushRadius = Math.max(0.5, options.size / 2);
const blurRadius = Math.max(1, Math.round(options.radius));
const padding = Math.ceil(blurRadius * 2.5);
const x1 = Math.max(0, Math.floor(Math.min(options.from.x, options.to.x) - brushRadius - padding));
const y1 = Math.max(0, Math.floor(Math.min(options.from.y, options.to.y) - brushRadius - padding));
const x2 = Math.min(target.width, Math.ceil(Math.max(options.from.x, options.to.x) + brushRadius + padding));
const y2 = Math.min(target.height, Math.ceil(Math.max(options.from.y, options.to.y) + brushRadius + padding));
const width = x2 - x1;
const height = y2 - y1;
if (width <= 0 || height <= 0) return;
const scratchWidth = width + padding * 2;
const scratchHeight = height + padding * 2;
resizeCanvas(target.featherSource, scratchWidth, scratchHeight);
resizeCanvas(target.featherBlur, scratchWidth, scratchHeight);
const sourceContext = target.featherSource.getContext("2d");
const blurContext = target.featherBlur.getContext("2d");
if (!sourceContext || !blurContext) return;
sourceContext.clearRect(0, 0, scratchWidth, scratchHeight);
sourceContext.drawImage(target.canvas, x1, y1, width, height, padding, padding, width, height);
blurContext.clearRect(0, 0, scratchWidth, scratchHeight);
blurContext.save();
blurContext.filter = `blur(${blurRadius}px)`;
blurContext.drawImage(target.featherSource, 0, 0);
blurContext.restore();
const original = target.context.getImageData(x1, y1, width, height);
const blurred = blurContext.getImageData(padding, padding, width, height);
blendFeatherPatch(original.data, blurred.data, width, height, { x: x1, y: y1 }, options);
target.context.putImageData(original, x1, y1);
}
export function blendFeatherPatch(
original: Uint8ClampedArray,
blurred: Uint8ClampedArray,
width: number,
height: number,
origin: { x: number; y: number },
options: { from: { x: number; y: number }; to: { x: number; y: number }; size: number; strength: number },
) {
const brushRadius = Math.max(0.5, options.size / 2);
const strength = Math.max(0.01, Math.min(1, options.strength / 100));
for (let y = 0; y < height; y += 1) {
for (let x = 0; x < width; x += 1) {
const distance = distanceToSegment(origin.x + x + 0.5, origin.y + y + 0.5, options.from, options.to);
if (distance >= brushRadius) continue;
const normalized = distance / brushRadius;
const coverage = 1 - smoothstep(0.72, 1, normalized);
const mix = coverage * strength;
const index = (y * width + x) * 4;
for (let channel = 0; channel < 4; channel += 1) {
const current = original[index + channel] ?? 0;
const feathered = blurred[index + channel] ?? 0;
original[index + channel] = Math.round(current + (feathered - current) * mix);
}
}
}
}
export function brushSurfaceDataUrl(surface: BrushSurface) { try { return internal(surface).canvas.toDataURL("image/png"); } catch { return undefined; } }
@@ -27,3 +109,22 @@ export function scheduleFrame(callback: () => void) { return requestAnimationFra
export function cancelFrame(id: number) { cancelAnimationFrame(id); }
function internal(surface: BrushSurface) { return surface as InternalSurface; }
function loadImage(source: string) { return new Promise<HTMLImageElement>((resolve, reject) => { const image = new Image(); image.onload = () => resolve(image); image.onerror = () => reject(new Error("Failed to load raster layer")); image.src = source; }); }
function resizeCanvas(canvas: HTMLCanvasElement, width: number, height: number) {
if (canvas.width !== width) canvas.width = width;
if (canvas.height !== height) canvas.height = height;
}
function distanceToSegment(x: number, y: number, from: { x: number; y: number }, to: { x: number; y: number }) {
const dx = to.x - from.x;
const dy = to.y - from.y;
const lengthSquared = dx * dx + dy * dy;
if (lengthSquared <= 0.0001) return Math.hypot(x - from.x, y - from.y);
const amount = Math.max(0, Math.min(1, ((x - from.x) * dx + (y - from.y) * dy) / lengthSquared));
return Math.hypot(x - (from.x + dx * amount), y - (from.y + dy * amount));
}
function smoothstep(edge0: number, edge1: number, value: number) {
const amount = Math.max(0, Math.min(1, (value - edge0) / Math.max(0.0001, edge1 - edge0)));
return amount * amount * (3 - 2 * amount);
}

View File

@@ -0,0 +1,24 @@
import { describe, expect, test } from "bun:test";
import { resolveLayerDrawImage, resolveTextLayerDraw } from "./exportArtboardPng";
describe("artboard PNG export", () => {
test("draws only cropped source pixels at their retained document position", () => {
const draw = resolveLayerDrawImage({
id: "layer", type: "raster", name: "Layer", visible: true, locked: false, opacity: 1, assetId: "asset",
sourceRect: { x: 10, y: 5, w: 40, h: 20 },
transform: { position: { x: 100, y: 50 }, scale: { x: 2, y: 3 }, rotation: 0 },
}, { w: 200, h: 100 });
expect(draw).toEqual({
source: { x: 10, y: 5, w: 40, h: 20 },
destination: { x: 120, y: 65, w: 80, h: 60 },
});
});
test("uses deterministic text data centered for renderer rotation parity", () => {
const draw = resolveTextLayerDraw({ id: "text", type: "text", name: "Text", visible: true, locked: false, opacity: 1, transform: { position: { x: 10, y: 20 }, scale: { x: 2, y: 3 }, rotation: 1 }, content: "AB", style: { fontFamily: "Arial", fontSize: 10, fontWeight: 400, fontStyle: "normal", color: "#ffffff", alignment: "center", lineHeight: 1.2 } });
expect(draw.size).toEqual({ w: 10.8, h: 12 });
expect(draw.center).toEqual({ x: 20.8, y: 38 });
expect(draw.textX).toBe(0);
});
});

View File

@@ -2,9 +2,20 @@ import type { Artboard } from "@core/artboard";
import type { Asset } from "@core/asset";
import type { Rect } from "@core/geometry";
import type { Layer } from "@core/layer";
import type { Size } from "@core/geometry";
import { getLayerMask } from "@core/layer-mask-utils";
import { measureTextLayer } from "@core/text-layer";
export async function downloadArtboardPng(artboard: Artboard, assets: readonly Asset[]) {
const canvas = await renderArtboardCanvas(artboard, assets);
const url = canvas.toDataURL("image/png");
const link = document.createElement("a");
link.href = url;
link.download = `${safeFilename(artboard.name)}.png`;
link.click();
}
export async function renderArtboardCanvas(artboard: Artboard, assets: readonly Asset[]): Promise<HTMLCanvasElement> {
const width = Math.max(1, Math.round(artboard.bounds.w));
const height = Math.max(1, Math.round(artboard.bounds.h));
const canvas = document.createElement("canvas");
@@ -25,11 +36,7 @@ export async function downloadArtboardPng(artboard: Artboard, assets: readonly A
for (const layer of renderStack(artboard.layers)) await drawLayer(context, layer, artboard.layers, assets, artboard.bounds, { maskLayerIds });
context.restore();
const url = canvas.toDataURL("image/png");
const link = document.createElement("a");
link.href = url;
link.download = `${safeFilename(artboard.name)}.png`;
link.click();
return canvas;
}
async function drawLayer(
@@ -54,7 +61,24 @@ async function drawLayer(
context.globalAlpha *= layer.opacity;
if (layer.type === "group") {
for (const child of renderStack(layer.children)) await drawLayer(context, child, layerTree, assets, artboardBounds, options);
const groupCanvas = createArtboardCanvas(artboardBounds);
const groupContext = translatedContext(groupCanvas, artboardBounds);
const groupMaskIds = collectMaskLayerIds(layer.children);
for (const child of renderStack(layer.children)) await drawLayer(groupContext, child, layer.children, assets, artboardBounds, { maskLayerIds: groupMaskIds });
groupContext.restore();
context.drawImage(groupCanvas, artboardBounds.x, artboardBounds.y);
context.restore();
return;
}
if (layer.type === "adjustment") {
applyAdjustmentToCanvas(context, layer.adjustment, layer.opacity);
context.restore();
return;
}
if (layer.type === "text") {
drawTextLayer(context, layer);
context.restore();
return;
}
@@ -66,16 +90,76 @@ async function drawLayer(
}
const image = await loadImage(asset.source);
context.drawImage(
image,
layer.transform.position.x,
layer.transform.position.y,
asset.intrinsicSize.w * layer.transform.scale.x,
asset.intrinsicSize.h * layer.transform.scale.y,
);
const draw = resolveLayerDrawImage(layer, asset.intrinsicSize);
context.drawImage(image, draw.source.x, draw.source.y, draw.source.w, draw.source.h,
draw.destination.x, draw.destination.y, draw.destination.w, draw.destination.h);
context.restore();
}
export function drawTextLayer(context: CanvasRenderingContext2D, layer: Extract<Layer, { type: "text" }>) {
const draw = resolveTextLayerDraw(layer);
const { size } = draw;
context.translate(draw.center.x, draw.center.y);
context.rotate(layer.transform.rotation);
context.scale(layer.transform.scale.x, layer.transform.scale.y);
context.fillStyle = layer.style.color;
context.font = `${layer.style.fontStyle} ${layer.style.fontWeight} ${layer.style.fontSize}px ${JSON.stringify(layer.style.fontFamily)}, Arial, sans-serif`;
context.textAlign = layer.style.alignment;
context.textBaseline = "alphabetic";
const x = draw.textX;
layer.content.split("\n").forEach((line, index) => context.fillText(line, x, (index + 0.82) * layer.style.fontSize * layer.style.lineHeight - size.h / 2));
}
export function resolveTextLayerDraw(layer: Extract<Layer, { type: "text" }>) {
const size = measureTextLayer(layer);
return {
size,
center: { x: layer.transform.position.x + size.w * layer.transform.scale.x / 2, y: layer.transform.position.y + size.h * layer.transform.scale.y / 2 },
textX: (layer.style.alignment === "left" ? 0 : layer.style.alignment === "center" ? size.w / 2 : size.w) - size.w / 2,
};
}
export function resolveLayerDrawImage(layer: Extract<Layer, { type: "image" | "raster" }>, intrinsicSize: Size) {
const source = layer.sourceRect ?? { x: 0, y: 0, ...intrinsicSize };
return {
source,
destination: {
x: layer.transform.position.x + source.x * layer.transform.scale.x,
y: layer.transform.position.y + source.y * layer.transform.scale.y,
w: source.w * layer.transform.scale.x,
h: source.h * layer.transform.scale.y,
},
};
}
export function adjustPixel(rgb: readonly [number, number, number], adjustment: Extract<Layer, { type: "adjustment" }>["adjustment"]): [number, number, number] {
let [red, green, blue] = rgb.map((value) => value / 255) as [number, number, number];
red += adjustment.brightness + adjustment.colorBalance.red;
green += adjustment.brightness + adjustment.colorBalance.green;
blue += adjustment.brightness + adjustment.colorBalance.blue;
const contrast = 1 + adjustment.contrast;
red = (red - 0.5) * contrast + 0.5;
green = (green - 0.5) * contrast + 0.5;
blue = (blue - 0.5) * contrast + 0.5;
const luminance = red * 0.2126 + green * 0.7152 + blue * 0.0722;
const saturation = 1 + adjustment.saturation;
return [red, green, blue].map((value) => Math.round(Math.max(0, Math.min(1, luminance + (value - luminance) * saturation)) * 255)) as [number, number, number];
}
function applyAdjustmentToCanvas(context: CanvasRenderingContext2D, adjustment: Extract<Layer, { type: "adjustment" }>["adjustment"], opacity: number) {
if (opacity <= 0) return;
const { width, height } = context.canvas;
const image = context.getImageData(0, 0, width, height);
for (let index = 0; index < image.data.length; index += 4) {
const original: [number, number, number] = [image.data[index]!, image.data[index + 1]!, image.data[index + 2]!];
const adjusted = adjustPixel(original, adjustment);
image.data[index] = Math.round(original[0] + (adjusted[0] - original[0]) * opacity);
image.data[index + 1] = Math.round(original[1] + (adjusted[1] - original[1]) * opacity);
image.data[index + 2] = Math.round(original[2] + (adjusted[2] - original[2]) * opacity);
}
context.putImageData(image, 0, 0);
}
async function drawMaskedLayer(
context: CanvasRenderingContext2D,
layer: Layer,

View File

@@ -8,8 +8,16 @@ export type BrowserImageFile = {
export async function decodeBrowserImageFile(file: File): Promise<BrowserImageFile | undefined> {
if (!file.type.startsWith("image/")) return undefined;
if (file.size > 128 * 1024 * 1024) throw new Error("Image file is too large. The import limit is 128 MB.");
const objectUrl = URL.createObjectURL(file);
let intrinsicSize: { w: number; h: number };
try {
intrinsicSize = await loadImageSize(objectUrl);
assertCanvasRasterSize(intrinsicSize.w, intrinsicSize.h, "Imported image");
} finally {
URL.revokeObjectURL(objectUrl);
}
const source = await fileToDataUrl(file);
const intrinsicSize = await loadImageSize(source);
return { name: file.name, mimeType: file.type, source, intrinsicSize, release: () => undefined };
}
@@ -30,3 +38,4 @@ function loadImageSize(source: string): Promise<{ w: number; h: number }> {
image.src = source;
});
}
import { assertCanvasRasterSize } from "./rasterLimits";

View File

@@ -1,6 +1,6 @@
import { blurMaskValues, despeckleMaskValues, dilateMaskValues, erodeMaskValues, maskValueFromRgba } from "./maskRaster";
export type MagicWandRasterSettings = { tolerance: number; feather: number; choke: number; despeckle: number; contiguous: boolean; mode: "replace" | "add" | "subtract" };
export type MagicWandRasterSettings = { tolerance: number; feather: number; choke: number; despeckle: number; contiguous: boolean; mode: "replace" | "add" | "subtract"; target?: "visibility" | "inpaint" };
export async function createWandMask(source: string, existingMaskSource: string | undefined, width: number, height: number, startX: number, startY: number, settings: MagicWandRasterSettings) {
const canvas = document.createElement("canvas");
@@ -10,8 +10,9 @@ export async function createWandMask(source: string, existingMaskSource: string
context.drawImage(await loadImage(source), 0, 0, canvas.width, canvas.height);
const data = context.getImageData(0, 0, canvas.width, canvas.height);
const start = (startY * canvas.width + startX) * 4;
const key = [data.data[start] ?? 0, data.data[start + 1] ?? 0, data.data[start + 2] ?? 0];
const selected = settings.contiguous ? floodSelect(data, width, height, startX, startY, key, settings.tolerance) : globalSelect(data, key, settings.tolerance);
const key = rgbToLab(data.data[start] ?? 0, data.data[start + 1] ?? 0, data.data[start + 2] ?? 0);
const keyAlpha = data.data[start + 3] ?? 255;
const selected = settings.contiguous ? floodSelect(data, width, height, startX, startY, key, keyAlpha, settings.tolerance) : globalSelect(data, key, keyAlpha, settings.tolerance);
let values: Uint8ClampedArray<ArrayBufferLike> = toValues(selected);
if (settings.despeckle > 0) values = despeckleMaskValues(values, width, height, Math.round(settings.despeckle));
if (settings.choke > 0) values = erodeMaskValues(values, width, height, Math.round(settings.choke));
@@ -19,20 +20,31 @@ export async function createWandMask(source: string, existingMaskSource: string
if (settings.feather > 0) values = blurMaskValues(values, width, height, Math.round(settings.feather));
const existing = existingMaskSource ? await loadMask(existingMaskSource, width, height) : undefined;
for (let pixel = 0; pixel < values.length; pixel++) {
const current = existing?.[pixel] ?? 255; const selectedValue = values[pixel] ?? 0;
const alpha = settings.mode === "add" ? Math.min(current, 255 - selectedValue) : settings.mode === "subtract" ? Math.max(current, selectedValue) : 255 - selectedValue;
const current = existing?.[pixel] ?? (settings.target === "inpaint" ? 0 : 255); const selectedValue = values[pixel] ?? 0;
const alpha = settings.target === "inpaint"
? settings.mode === "add" ? Math.max(current, selectedValue) : settings.mode === "subtract" ? Math.min(current, 255 - selectedValue) : selectedValue
: settings.mode === "add" ? Math.min(current, 255 - selectedValue) : settings.mode === "subtract" ? Math.max(current, selectedValue) : 255 - selectedValue;
const index = pixel * 4; data.data[index] = data.data[index + 1] = data.data[index + 2] = 255; data.data[index + 3] = alpha;
}
context.putImageData(data, 0, 0); return canvas.toDataURL("image/png");
}
function floodSelect(data: ImageData, width: number, height: number, x: number, y: number, key: number[], tolerance: number) {
function floodSelect(data: ImageData, width: number, height: number, x: number, y: number, key: readonly number[], keyAlpha: number, tolerance: number) {
const result = new Uint8Array(width * height); const queue: Array<[number, number]> = [[x, y]];
while (queue.length) { const [px, py] = queue.pop()!; if (px < 0 || py < 0 || px >= width || py >= height) continue; const i = py * width + px; if (result[i] || !matches(data, i, key, tolerance)) continue; result[i] = 1; queue.push([px + 1, py], [px - 1, py], [px, py + 1], [px, py - 1]); }
while (queue.length) { const [px, py] = queue.pop()!; if (px < 0 || py < 0 || px >= width || py >= height) continue; const i = py * width + px; if (result[i] || !matches(data, i, key, keyAlpha, tolerance)) continue; result[i] = 1; queue.push([px + 1, py], [px - 1, py], [px, py + 1], [px, py - 1]); }
return result;
}
function globalSelect(data: ImageData, key: number[], tolerance: number) { const result = new Uint8Array(data.width * data.height); for (let i = 0; i < result.length; i++) if (matches(data, i, key, tolerance)) result[i] = 1; return result; }
function matches(data: ImageData, pixel: number, key: number[], tolerance: number) { const i = pixel * 4; return Math.hypot((data.data[i] ?? 0) - (key[0] ?? 0), (data.data[i + 1] ?? 0) - (key[1] ?? 0), (data.data[i + 2] ?? 0) - (key[2] ?? 0)) <= tolerance; }
function globalSelect(data: ImageData, key: readonly number[], keyAlpha: number, tolerance: number) { const result = new Uint8Array(data.width * data.height); for (let i = 0; i < result.length; i++) if (matches(data, i, key, keyAlpha, tolerance)) result[i] = 1; return result; }
function matches(data: ImageData, pixel: number, key: readonly number[], keyAlpha: number, tolerance: number) { const i = pixel * 4; const sample = rgbToLab(data.data[i] ?? 0, data.data[i + 1] ?? 0, data.data[i + 2] ?? 0); const colorDistance = Math.hypot((sample[0] ?? 0) - (key[0] ?? 0), (sample[1] ?? 0) - (key[1] ?? 0), (sample[2] ?? 0) - (key[2] ?? 0)); const alphaDistance = Math.abs((data.data[i + 3] ?? 255) - keyAlpha) / 2.55; return Math.hypot(colorDistance, alphaDistance) <= tolerance * 0.45; }
function rgbToLab(red: number, green: number, blue: number): [number, number, number] {
const linear = [red, green, blue].map((value) => { const channel = value / 255; return channel <= 0.04045 ? channel / 12.92 : ((channel + 0.055) / 1.055) ** 2.4; });
const x = ((linear[0] ?? 0) * 0.4124 + (linear[1] ?? 0) * 0.3576 + (linear[2] ?? 0) * 0.1805) / 0.95047;
const y = ((linear[0] ?? 0) * 0.2126 + (linear[1] ?? 0) * 0.7152 + (linear[2] ?? 0) * 0.0722);
const z = ((linear[0] ?? 0) * 0.0193 + (linear[1] ?? 0) * 0.1192 + (linear[2] ?? 0) * 0.9505) / 1.08883;
const f = (value: number) => value > 0.008856 ? Math.cbrt(value) : 7.787 * value + 16 / 116;
return [116 * f(y) - 16, 500 * (f(x) - f(y)), 200 * (f(y) - f(z))];
}
function toValues(selected: Uint8Array) { const values = new Uint8ClampedArray(selected.length); for (let i = 0; i < selected.length; i++) values[i] = selected[i] ? 255 : 0; return values; }
async function loadMask(source: string, width: number, height: number) { const canvas = document.createElement("canvas"); canvas.width = width; canvas.height = height; const context = canvas.getContext("2d"); if (!context) return undefined; context.drawImage(await loadImage(source), 0, 0, width, height); const data = context.getImageData(0, 0, width, height); const values = new Uint8ClampedArray(width * height); for (let i = 0; i < values.length; i++) values[i] = maskValueFromRgba(data.data, i * 4); return values; }
function loadImage(source: string) { return new Promise<HTMLImageElement>((resolve, reject) => { const image = new Image(); image.onload = () => resolve(image); image.onerror = () => reject(new Error("Failed to load image")); image.src = source; }); }

View File

@@ -1,5 +1,5 @@
import { describe, expect, test } from "bun:test";
import { cropMaskValuesToRgba, expandRectWithinBounds, invertMaskValues } from "./maskRaster";
import { blurMaskValues, cropMaskValuesToRgba, dilateMaskValues, erodeMaskValues, expandRectWithinBounds, invertMaskValues, limitMaskValues } from "./maskRaster";
describe("mask raster utilities", () => {
test("exports the normalized drawn mask without filling the whole crop", () => {
@@ -28,6 +28,22 @@ describe("mask raster utilities", () => {
h: 64,
});
});
test("uses separable mask kernels without changing square-kernel semantics", () => {
const values = new Uint8ClampedArray([0, 0, 0, 0, 255, 0, 0, 0, 0]);
expect([...dilateMaskValues(values, 3, 3, 1)]).toEqual(new Array(9).fill(255));
expect([...erodeMaskValues(new Uint8ClampedArray(new Array(9).fill(255)), 3, 3, 1)]).toEqual(new Array(9).fill(255));
expect([...blurMaskValues(values, 3, 3, 1)]).toEqual(new Array(9).fill(28));
});
test("clips AI edit regions to a retained source crop", () => {
expect([...limitMaskValues(new Uint8ClampedArray(16).fill(255), 4, 4, { x: 1, y: 1, w: 2, h: 2 })]).toEqual([
0, 0, 0, 0,
0, 255, 255, 0,
0, 255, 255, 0,
0, 0, 0, 0,
]);
});
});
function activeRedPixels(rgba: Uint8ClampedArray) {

View File

@@ -1,4 +1,5 @@
import type { Rect } from "@core/geometry";
import { assertCanvasRasterSize, assertProcessingRasterSize } from "./rasterLimits";
export type MaskFill = "white" | "black" | "clear";
@@ -29,9 +30,11 @@ export type NormalizedMaskOptions = {
feather?: number;
blur?: number;
despeckle?: number;
limit?: Rect;
};
export async function createSolidMaskSource(width: number, height: number, fill: MaskFill): Promise<string> {
assertProcessingRasterSize(width, height, "Mask");
const canvas = createCanvas(width, height);
const context = require2dContext(canvas);
context.clearRect(0, 0, canvas.width, canvas.height);
@@ -48,6 +51,7 @@ export async function createSolidMaskSource(width: number, height: number, fill:
}
export async function applyMaskRasterOperation(source: string, width: number, height: number, operation: MaskRasterOperation): Promise<string> {
assertProcessingRasterSize(width, height, "Mask");
if (operation.type === "fill") return createSolidMaskSource(width, height, operation.fill);
const mask = await loadMaskValues(source, width, height);
@@ -55,7 +59,38 @@ export async function applyMaskRasterOperation(source: string, width: number, he
return maskValuesToDataUrl(next, mask.width, mask.height);
}
export async function applyPolygonMask(source: string, width: number, height: number, points: readonly { x: number; y: number }[], mode: "replace" | "add" | "subtract"): Promise<string> {
assertProcessingRasterSize(width, height, "Mask");
if (points.length < 3) return source;
const canvas = await loadImageCanvas(source, width, height);
const context = require2dContext(canvas);
if (mode === "replace") context.clearRect(0, 0, canvas.width, canvas.height);
context.save();
context.globalCompositeOperation = mode === "subtract" ? "destination-out" : "source-over";
context.fillStyle = "#ffffff";
context.beginPath();
context.moveTo(points[0]?.x ?? 0, points[0]?.y ?? 0);
for (let index = 1; index < points.length; index += 1) context.lineTo(points[index]?.x ?? 0, points[index]?.y ?? 0);
context.closePath();
context.fill();
context.restore();
return canvas.toDataURL("image/png");
}
export async function mergeMaskSources(existingSource: string, selectedSource: string, width: number, height: number, mode: "replace" | "add" | "subtract"): Promise<string> {
assertProcessingRasterSize(width, height, "Mask");
const [existing, selected] = await Promise.all([loadMaskValues(existingSource, width, height), loadMaskValues(selectedSource, width, height)]);
const values = new Uint8ClampedArray(width * height);
for (let pixel = 0; pixel < values.length; pixel += 1) {
const current = existing.values[pixel] ?? 0;
const choice = selected.values[pixel] ?? 0;
values[pixel] = mode === "add" ? Math.max(current, choice) : mode === "subtract" ? Math.min(current, 255 - choice) : choice;
}
return maskValuesToDataUrl(values, width, height);
}
export async function analyzeMaskSource(source: string, width: number, height: number): Promise<MaskAnalysis> {
assertProcessingRasterSize(width, height, "Mask");
const mask = await loadMaskValues(source, width, height);
const revealed = analyzeValues(mask.values, mask.width, mask.height, false);
const hiddenValues = invertMaskValues(mask.values);
@@ -74,8 +109,10 @@ export async function analyzeMaskSource(source: string, width: number, height: n
}
export async function createNormalizedMaskSource(source: string, width: number, height: number, options: NormalizedMaskOptions): Promise<{ source: string; values: Uint8ClampedArray; bounds?: Rect }> {
assertProcessingRasterSize(width, height, "Mask");
const mask = await loadMaskValues(source, width, height);
let values = options.polarity === "hidden" ? invertMaskValues(mask.values) : new Uint8ClampedArray(mask.values);
if (options.limit) values = limitMaskValues(values, mask.width, mask.height, options.limit);
const despeckle = Math.round(clampNumber(options.despeckle ?? 0, 0, 64));
const expand = Math.round(clampNumber(options.expand ?? 0, -256, 256));
@@ -92,8 +129,19 @@ export async function createNormalizedMaskSource(source: string, width: number,
return { source: maskValuesToDataUrl(values, mask.width, mask.height), values, bounds: analysis.bounds };
}
export function limitMaskValues(values: Uint8ClampedArray, width: number, height: number, limit: Rect): Uint8ClampedArray {
const next = new Uint8ClampedArray(values.length);
const x1 = Math.max(0, Math.floor(limit.x));
const y1 = Math.max(0, Math.floor(limit.y));
const x2 = Math.min(width, Math.ceil(limit.x + limit.w));
const y2 = Math.min(height, Math.ceil(limit.y + limit.h));
for (let y = y1; y < y2; y += 1) for (let x = x1; x < x2; x += 1) next[y * width + x] = values[y * width + x] ?? 0;
return next;
}
export async function loadImageCanvas(source: string, width?: number, height?: number): Promise<HTMLCanvasElement> {
const image = await loadImage(source);
assertCanvasRasterSize(width ?? image.naturalWidth, height ?? image.naturalHeight);
const canvas = createCanvas(width ?? image.naturalWidth, height ?? image.naturalHeight);
const context = require2dContext(canvas);
context.clearRect(0, 0, canvas.width, canvas.height);
@@ -108,6 +156,7 @@ export async function imageSourceToPngDataUrl(source: string): Promise<string> {
}
export function cropCanvas(sourceCanvas: HTMLCanvasElement, crop: Rect, outputWidth = crop.w, outputHeight = crop.h): string {
assertProcessingRasterSize(outputWidth, outputHeight, "Crop");
const canvas = createCanvas(outputWidth, outputHeight);
const context = require2dContext(canvas);
context.clearRect(0, 0, outputWidth, outputHeight);
@@ -115,7 +164,35 @@ export function cropCanvas(sourceCanvas: HTMLCanvasElement, crop: Rect, outputWi
return canvas.toDataURL("image/png");
}
export function sampleDocumentCanvasInLayerSpace(
documentCanvas: HTMLCanvasElement,
artboardBounds: Rect,
layer: { transform: { position: { x: number; y: number }; scale: { x: number; y: number }; rotation: number }; sourceRect?: Rect },
intrinsicSize: { w: number; h: number },
crop: Rect,
outputWidth = crop.w,
outputHeight = crop.h,
): string {
assertProcessingRasterSize(outputWidth, outputHeight, "Context crop");
const canvas = createCanvas(outputWidth, outputHeight);
const context = require2dContext(canvas);
const source = layer.sourceRect ?? { x: 0, y: 0, ...intrinsicSize };
const center = {
x: layer.transform.position.x + (source.x + source.w / 2) * layer.transform.scale.x,
y: layer.transform.position.y + (source.y + source.h / 2) * layer.transform.scale.y,
};
context.translate(-crop.x, -crop.y);
context.scale(1 / Math.max(0.0001, layer.transform.scale.x), 1 / Math.max(0.0001, layer.transform.scale.y));
context.translate(-layer.transform.position.x, -layer.transform.position.y);
context.translate(center.x, center.y);
context.rotate(-layer.transform.rotation);
context.translate(-center.x, -center.y);
context.drawImage(documentCanvas, artboardBounds.x, artboardBounds.y);
return canvas.toDataURL("image/png");
}
export function cropMaskValuesToDataUrl(values: Uint8ClampedArray, width: number, height: number, crop: Rect, outputWidth = crop.w, outputHeight = crop.h): string {
assertProcessingRasterSize(outputWidth, outputHeight, "Mask crop");
const canvas = createCanvas(outputWidth, outputHeight);
const context = require2dContext(canvas);
const imageData = context.createImageData(outputWidth, outputHeight);
@@ -203,51 +280,52 @@ export function invertMaskValues(values: Uint8ClampedArray): Uint8ClampedArray {
export function erodeMaskValues(values: Uint8ClampedArray, width: number, height: number, radius: number): Uint8ClampedArray {
const safeRadius = Math.round(clampNumber(radius, 0, 256));
if (safeRadius <= 0) return new Uint8ClampedArray(values);
const next = new Uint8ClampedArray(values.length);
for (let y = 0; y < height; y += 1) {
for (let x = 0; x < width; x += 1) {
let value = 255;
for (let oy = -safeRadius; oy <= safeRadius; oy += 1) {
for (let ox = -safeRadius; ox <= safeRadius; ox += 1) value = Math.min(value, values[clampInt(y + oy, 0, height - 1) * width + clampInt(x + ox, 0, width - 1)] ?? 0);
}
next[y * width + x] = value;
}
}
return next;
return separableExtrema(values, width, height, safeRadius, Math.min, 255);
}
export function dilateMaskValues(values: Uint8ClampedArray, width: number, height: number, radius: number): Uint8ClampedArray {
const safeRadius = Math.round(clampNumber(radius, 0, 256));
if (safeRadius <= 0) return new Uint8ClampedArray(values);
const next = new Uint8ClampedArray(values.length);
for (let y = 0; y < height; y += 1) {
for (let x = 0; x < width; x += 1) {
let value = 0;
for (let oy = -safeRadius; oy <= safeRadius; oy += 1) {
for (let ox = -safeRadius; ox <= safeRadius; ox += 1) value = Math.max(value, values[clampInt(y + oy, 0, height - 1) * width + clampInt(x + ox, 0, width - 1)] ?? 0);
}
next[y * width + x] = value;
}
}
return next;
return separableExtrema(values, width, height, safeRadius, Math.max, 0);
}
export function blurMaskValues(values: Uint8ClampedArray, width: number, height: number, radius: number): Uint8ClampedArray {
const safeRadius = Math.round(clampNumber(radius, 0, 256));
if (safeRadius <= 0) return new Uint8ClampedArray(values);
const horizontal = new Float64Array(values.length);
const next = new Uint8ClampedArray(values.length);
const span = safeRadius * 2 + 1;
for (let y = 0; y < height; y += 1) {
let total = 0;
for (let ox = -safeRadius; ox <= safeRadius; ox += 1) total += values[y * width + clampInt(ox, 0, width - 1)] ?? 0;
for (let x = 0; x < width; x += 1) {
horizontal[y * width + x] = total / span;
total += (values[y * width + clampInt(x + safeRadius + 1, 0, width - 1)] ?? 0) - (values[y * width + clampInt(x - safeRadius, 0, width - 1)] ?? 0);
}
}
for (let x = 0; x < width; x += 1) {
let total = 0;
let count = 0;
for (let oy = -safeRadius; oy <= safeRadius; oy += 1) {
for (let ox = -safeRadius; ox <= safeRadius; ox += 1) {
total += values[clampInt(y + oy, 0, height - 1) * width + clampInt(x + ox, 0, width - 1)] ?? 0;
count += 1;
for (let oy = -safeRadius; oy <= safeRadius; oy += 1) total += horizontal[clampInt(oy, 0, height - 1) * width + x] ?? 0;
for (let y = 0; y < height; y += 1) {
next[y * width + x] = Math.round(total / span);
total += (horizontal[clampInt(y + safeRadius + 1, 0, height - 1) * width + x] ?? 0) - (horizontal[clampInt(y - safeRadius, 0, height - 1) * width + x] ?? 0);
}
}
next[y * width + x] = Math.round(total / count);
return next;
}
function separableExtrema(values: Uint8ClampedArray, width: number, height: number, radius: number, combine: (a: number, b: number) => number, initial: number) {
const horizontal = new Uint8ClampedArray(values.length);
const next = new Uint8ClampedArray(values.length);
for (let y = 0; y < height; y += 1) for (let x = 0; x < width; x += 1) {
let value = initial;
for (let offset = -radius; offset <= radius; offset += 1) value = combine(value, values[y * width + clampInt(x + offset, 0, width - 1)] ?? 0);
horizontal[y * width + x] = value;
}
for (let y = 0; y < height; y += 1) for (let x = 0; x < width; x += 1) {
let value = initial;
for (let offset = -radius; offset <= radius; offset += 1) value = combine(value, horizontal[clampInt(y + offset, 0, height - 1) * width + x] ?? 0);
next[y * width + x] = value;
}
return next;
}

View File

@@ -0,0 +1,55 @@
import type { RecoveryStorage } from "@platform/projectStorage";
export const browserRecoveryStorage: RecoveryStorage = {
async read(key) {
return request<string | undefined>("readonly", (store) => store.get(key)).then((value) => value ?? null);
},
async write(key, value) {
await request("readwrite", (store) => store.put(value, key));
},
async remove(key) {
await request("readwrite", (store) => store.delete(key));
},
};
const RECOVERY_DATABASE = "image-studio";
const RECOVERY_STORE = "recovery";
function openRecoveryDatabase(): Promise<IDBDatabase> {
return new Promise((resolve, reject) => {
const open = indexedDB.open(RECOVERY_DATABASE, 1);
open.onupgradeneeded = () => {
if (!open.result.objectStoreNames.contains(RECOVERY_STORE)) open.result.createObjectStore(RECOVERY_STORE);
};
open.onsuccess = () => resolve(open.result);
open.onerror = () => reject(open.error ?? new Error("Could not open recovery storage."));
});
}
async function request<T>(mode: IDBTransactionMode, operation: (store: IDBObjectStore) => IDBRequest<T>): Promise<T> {
const database = await openRecoveryDatabase();
return new Promise((resolve, reject) => {
const transaction = database.transaction(RECOVERY_STORE, mode);
const databaseRequest = operation(transaction.objectStore(RECOVERY_STORE));
databaseRequest.onsuccess = () => resolve(databaseRequest.result);
databaseRequest.onerror = () => reject(databaseRequest.error ?? new Error("Recovery storage operation failed."));
transaction.oncomplete = () => database.close();
transaction.onerror = () => {
database.close();
reject(transaction.error ?? new Error("Recovery storage transaction failed."));
};
});
}
export function downloadProjectFile(source: string, fileName: string): void {
const url = URL.createObjectURL(new Blob([source], { type: "application/json" }));
const link = document.createElement("a");
link.href = url;
link.download = fileName;
link.click();
URL.revokeObjectURL(url);
}
export function readProjectFile(file: File): Promise<string> {
return file.text();
}

View File

@@ -0,0 +1,20 @@
const maxCanvasDimension = 16_384;
const maxCanvasPixels = 64 * 1024 * 1024;
const maxProcessingPixels = 24 * 1024 * 1024;
export function assertCanvasRasterSize(width: number, height: number, label = "Image") {
assertRasterSize(width, height, maxCanvasPixels, label);
}
export function assertProcessingRasterSize(width: number, height: number, label = "Image") {
assertRasterSize(width, height, maxProcessingPixels, label);
}
function assertRasterSize(width: number, height: number, maxPixels: number, label: string) {
const w = Math.round(width);
const h = Math.round(height);
if (!Number.isSafeInteger(w) || !Number.isSafeInteger(h) || w <= 0 || h <= 0) throw new Error(`${label} dimensions are invalid.`);
if (w > maxCanvasDimension || h > maxCanvasDimension || w * h > maxPixels) {
throw new Error(`${label} is too large (${w} x ${h}). Maximum processing size is ${maxCanvasDimension} px per side and ${Math.floor(maxPixels / 1_000_000)} megapixels.`);
}
}

View File

@@ -4,8 +4,40 @@ export async function fetchGenerationOptions(): Promise<unknown> {
return response.json() as Promise<unknown>;
}
export async function requestGeneration(body: unknown): Promise<{ source: string; mimeType: string }> {
const response = await fetch("/api/comfy/generate", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) });
export type GenerationResult = { source: string; mimeType: string; seed: number };
export async function requestSemanticSelection(body: { inputImage: string; x: number; y: number; model?: string }, signal?: AbortSignal): Promise<{ source: string; mimeType: string }> {
const response = await fetch("/api/comfy/segment", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body), signal });
if (!response.ok) throw new Error(await response.text());
return response.json() as Promise<{ source: string; mimeType: string }>;
}
export async function requestGeneration(body: unknown, signal?: AbortSignal, onProgress?: (progress: number, detail: string) => void): Promise<{ results: GenerationResult[] }> {
const response = await fetch("/api/comfy/generate", { method: "POST", headers: { "content-type": "application/json", accept: "application/x-ndjson" }, body: JSON.stringify(body), signal });
if (!response.ok) throw new Error(await response.text());
const contentType = response.headers.get("content-type") ?? "";
if (contentType.includes("application/x-ndjson") && response.body) {
const reader = response.body.getReader();
const decoder = new TextDecoder();
let pending = "";
while (true) {
const { value, done } = await reader.read();
pending += decoder.decode(value, { stream: !done });
const lines = pending.split("\n");
pending = lines.pop() ?? "";
for (const line of lines) {
if (!line.trim()) continue;
const event = JSON.parse(line) as { type: "progress"; progress: number; detail: string } | { type: "result"; results: GenerationResult[] } | { type: "error"; message: string };
if (event.type === "progress") onProgress?.(event.progress, event.detail);
if (event.type === "error") throw new Error(event.message);
if (event.type === "result") return { results: event.results };
}
if (done) break;
}
throw new Error("Generation stream ended without results");
}
const payload = await response.json() as { results?: GenerationResult[]; source?: string; mimeType?: string };
if (payload.results?.length) return { results: payload.results };
if (payload.source) return { results: [{ source: payload.source, mimeType: payload.mimeType ?? "image/png", seed: 0 }] };
throw new Error("Generation returned no results");
}

View File

@@ -0,0 +1,5 @@
export type RecoveryStorage = {
read(key: string): Promise<string | null>;
write(key: string, value: string): Promise<void>;
remove(key: string): Promise<void>;
};

111
renderer/adjustment-pass.ts Normal file
View File

@@ -0,0 +1,111 @@
import type { ColorAdjustment } from "@core/adjustment-layer";
import type { ScreenRect, WebGlRendererContext } from "./types";
export type AdjustmentPass = {
apply(adjustment: ColorAdjustment, opacity: number, clip: ScreenRect): void;
dispose(): void;
};
const vertexSource = `#version 300 es
in vec2 a_position;
void main(){ gl_Position=vec4(a_position,0.,1.); }`;
const fragmentSource = `#version 300 es
precision highp float;
uniform sampler2D u_pixels; uniform vec2 u_size; uniform float u_brightness,u_contrast,u_saturation,u_opacity; uniform vec3 u_balance;
out vec4 outColor;
void main(){ vec2 uv=gl_FragCoord.xy/u_size; vec4 original=texture(u_pixels,uv); vec3 c=original.rgb+vec3(u_brightness)+u_balance; c=(c-0.5)*(1.0+u_contrast)+0.5; float l=dot(c,vec3(.2126,.7152,.0722)); c=clamp(vec3(l)+(c-vec3(l))*(1.0+u_saturation),0.0,1.0); outColor=vec4(mix(original.rgb,c,u_opacity),original.a); }`;
/** Renderer-owned framebuffer adjustment resources, reused across layers/frames. */
export function createAdjustmentPass(context: WebGlRendererContext): AdjustmentPass {
const { gl, canvas } = context;
const program = createProgram(gl, vertexSource, fragmentSource);
const texture = gl.createTexture();
const buffer = gl.createBuffer();
if (!program || !texture || !buffer) throw new Error("Failed to create adjustment pass");
const position = gl.getAttribLocation(program, "a_position");
const size = requiredUniform(gl, program, "u_size");
const brightness = requiredUniform(gl, program, "u_brightness");
const contrast = requiredUniform(gl, program, "u_contrast");
const saturation = requiredUniform(gl, program, "u_saturation");
const opacityLocation = requiredUniform(gl, program, "u_opacity");
const balance = requiredUniform(gl, program, "u_balance");
const pixels = requiredUniform(gl, program, "u_pixels");
let allocatedWidth = 0;
let allocatedHeight = 0;
let disposed = false;
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([-1, -1, 1, -1, -1, 1, -1, 1, 1, -1, 1, 1]), gl.STATIC_DRAW);
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
return {
apply(adjustment, opacity, clip) {
if (disposed || canvas.width < 1 || canvas.height < 1) return;
gl.activeTexture(gl.TEXTURE0);
gl.bindTexture(gl.TEXTURE_2D, texture);
if (allocatedWidth !== canvas.width || allocatedHeight !== canvas.height) {
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, canvas.width, canvas.height, 0, gl.RGBA, gl.UNSIGNED_BYTE, null);
allocatedWidth = canvas.width;
allocatedHeight = canvas.height;
}
gl.copyTexSubImage2D(gl.TEXTURE_2D, 0, 0, 0, 0, 0, canvas.width, canvas.height);
gl.useProgram(program);
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
gl.enableVertexAttribArray(position);
gl.vertexAttribPointer(position, 2, gl.FLOAT, false, 0, 0);
gl.uniform1i(pixels, 0);
gl.uniform2f(size, canvas.width, canvas.height);
gl.uniform1f(brightness, adjustment.brightness);
gl.uniform1f(contrast, adjustment.contrast);
gl.uniform1f(saturation, adjustment.saturation);
gl.uniform1f(opacityLocation, Math.max(0, Math.min(1, opacity)));
gl.uniform3f(balance, adjustment.colorBalance.red, adjustment.colorBalance.green, adjustment.colorBalance.blue);
gl.scissor(clip.x, canvas.height - clip.y - clip.h, clip.w, clip.h);
gl.disable(gl.BLEND);
gl.drawArrays(gl.TRIANGLES, 0, 6);
gl.enable(gl.BLEND);
},
dispose() {
if (disposed) return;
disposed = true;
gl.deleteTexture(texture);
gl.deleteBuffer(buffer);
gl.deleteProgram(program);
},
};
}
function requiredUniform(gl: WebGL2RenderingContext, program: WebGLProgram, name: string) {
const location = gl.getUniformLocation(program, name);
if (!location) throw new Error(`Missing adjustment uniform: ${name}`);
return location;
}
function createProgram(gl: WebGL2RenderingContext, vertex: string, fragment: string) {
const vertexShader = compile(gl, gl.VERTEX_SHADER, vertex);
const fragmentShader = compile(gl, gl.FRAGMENT_SHADER, fragment);
if (!vertexShader || !fragmentShader) return undefined;
const program = gl.createProgram();
if (!program) return undefined;
gl.attachShader(program, vertexShader);
gl.attachShader(program, fragmentShader);
gl.linkProgram(program);
gl.deleteShader(vertexShader);
gl.deleteShader(fragmentShader);
if (gl.getProgramParameter(program, gl.LINK_STATUS)) return program;
gl.deleteProgram(program);
return undefined;
}
function compile(gl: WebGL2RenderingContext, type: number, source: string) {
const shader = gl.createShader(type);
if (!shader) return undefined;
gl.shaderSource(shader, source);
gl.compileShader(shader);
if (gl.getShaderParameter(shader, gl.COMPILE_STATUS)) return shader;
gl.deleteShader(shader);
return undefined;
}

View File

@@ -1,7 +1,7 @@
import type { ImageDocument } from "@core/document";
import type { Vec2D } from "@core/geometry";
import type { Layer } from "@core/layer";
import type { RasterLayer } from "@core/raster-layer";
import { getLayerMask } from "@core/layer-mask-utils";
import type { EditorState } from "@editor/state";
import type { RgbaColor, WebGlRendererContext } from "./types";
@@ -79,13 +79,13 @@ function drawPreviewPass(
}
function resolveBrushPreview(document: ImageDocument, editor: EditorState, canvas: HTMLCanvasElement) {
if (!editor.brushPreview || (editor.tools.activeTool !== "brush" && editor.tools.activeTool !== "eraser")) return undefined;
if (!editor.brushPreview || (editor.tools.activeTool !== "brush" && editor.tools.activeTool !== "eraser" && editor.tools.activeTool !== "feather")) return undefined;
if (editor.tools.interactionMode.type === "temporary-pan" || (editor.tools.interactionMode.type === "tool" && editor.tools.interactionMode.tool === "pan")) return undefined;
const layer = resolveBrushTargetLayer(document, editor);
if (!layer) return undefined;
const size = Math.max(1, editor.tools.brush.size);
const size = Math.max(1, editor.tools.activeTool === "feather" ? editor.tools.feather.size : editor.tools.brush.size);
const zoom = editor.viewport.zoom;
return {
center: documentPointToScreenPoint(canvas, editor.brushPreview.position, editor),
@@ -93,16 +93,19 @@ function resolveBrushPreview(document: ImageDocument, editor: EditorState, canva
x: Math.max(1, Math.abs(layer.transform.scale.x) * size * zoom * 0.5),
y: Math.max(1, Math.abs(layer.transform.scale.y) * size * zoom * 0.5),
},
hardness: Math.max(0, Math.min(1, editor.tools.brush.hardness / 100)),
hardness: editor.tools.activeTool === "feather" ? 0.72 : Math.max(0, Math.min(1, editor.tools.brush.hardness / 100)),
};
}
function resolveBrushTargetLayer(document: ImageDocument, editor: EditorState): RasterLayer | undefined {
function resolveBrushTargetLayer(document: ImageDocument, editor: EditorState): Extract<Layer, { type: "image" | "raster" }> | undefined {
const editingMask = Boolean(editor.maskEdit);
const layerId = editor.maskEdit?.maskLayerId ?? editor.selection.layerIds[0];
const layerId = editor.maskEdit?.kind === "inpaintRegion" ? editor.maskEdit.targetLayerId : editor.maskEdit?.maskLayerId ?? editor.selection.layerIds[0];
if (!layerId) return undefined;
const layer = findRasterLayer(document.artboards.flatMap((artboard) => artboard.layers), layerId);
const layers = document.artboards.flatMap((artboard) => artboard.layers);
const selectedLayer = findPaintableLayer(layers, layerId);
const maskLayerId = editor.tools.activeTool === "feather" && !editingMask && selectedLayer ? getLayerMask(selectedLayer)?.maskLayerId : undefined;
const layer = maskLayerId ? findPaintableLayer(layers, maskLayerId) : selectedLayer;
if (!layer || layer.locked || (!editingMask && !layer.visible)) return undefined;
return layer;
}
@@ -124,11 +127,11 @@ function previewVertices(center: Vec2D, radius: Vec2D) {
return new Float32Array([x1, y1, x2, y1, x1, y2, x1, y2, x2, y1, x2, y2]);
}
function findRasterLayer(layers: readonly Layer[], layerId: string): RasterLayer | undefined {
function findPaintableLayer(layers: readonly Layer[], layerId: string): Extract<Layer, { type: "image" | "raster" }> | undefined {
for (const layer of layers) {
if (layer.id === layerId && layer.type === "raster") return layer;
if (layer.id === layerId && (layer.type === "image" || layer.type === "raster")) return layer;
if (layer.type === "group") {
const child = findRasterLayer(layer.children, layerId);
const child = findPaintableLayer(layer.children, layerId);
if (child) return child;
}
}

View File

@@ -80,6 +80,8 @@ export function maskVisualizationModeValue(mode: MaskVisualizationMode) {
return 1;
case "hiddenOverlay":
return 2;
case "activeOverlay":
return 3;
}
}
@@ -119,7 +121,7 @@ function createMaskVisualizationProgram(gl: WebGL2RenderingContext): MaskVisuali
return;
}
float alpha = (1.0 - maskAlpha) * u_color.a;
float alpha = (u_mode == 3 ? maskAlpha : (1.0 - maskAlpha)) * u_color.a;
outColor = vec4(u_color.rgb * alpha, alpha);
}`,
);

View File

@@ -1,14 +1,17 @@
import { createMaskedProgram, createMaskRevealPreviewProgram, createProgram, createTintedProgram, getMaskVisualizationResources, maskVisualizationModeValue, type MaskVisualizationResources } from "./image-texture-programs";
import type { Asset } from "@core/asset";
import type { RgbaColor, ScreenRect, WebGlRendererContext } from "./types";
import type { Rect } from "@core/geometry";
import { rotatedRectBounds, rotatedRectCorners } from "./rotated-rect";
import { textureCoordinatesForCrop, textureCoordinatesForRect } from "./texture-coordinates";
export type MaskVisualizationMode = "blackWhite" | "alpha" | "hiddenOverlay";
export type MaskVisualizationMode = "blackWhite" | "alpha" | "hiddenOverlay" | "activeOverlay";
export type ImageTextureRenderer = {
syncAssets(assets: readonly Asset[]): void;
render(asset: Asset, rect: ScreenRect, clipRect?: ScreenRect, opacity?: number): boolean;
renderMasked(asset: Asset, rect: ScreenRect, maskAsset: Asset, maskRect: ScreenRect, clipRect?: ScreenRect, opacity?: number): boolean;
renderMaskRevealPreview(asset: Asset, rect: ScreenRect, maskAsset: Asset, maskRect: ScreenRect, opacity: number, clipRect?: ScreenRect, layerOpacity?: number): boolean;
syncAssets(assets: readonly Pick<Asset, "id">[]): void;
render(asset: Asset, rect: ScreenRect, clipRect?: ScreenRect, opacity?: number, rotation?: number, sourceRect?: Rect): boolean;
renderMasked(asset: Asset, rect: ScreenRect, maskAsset: Asset, maskRect: ScreenRect, clipRect?: ScreenRect, opacity?: number, rotation?: number, sourceRect?: Rect): boolean;
renderMaskRevealPreview(asset: Asset, rect: ScreenRect, maskAsset: Asset, maskRect: ScreenRect, opacity: number, clipRect?: ScreenRect, layerOpacity?: number, rotation?: number, sourceRect?: Rect): boolean;
renderMaskVisualization(maskAsset: Asset, maskRect: ScreenRect, mode: MaskVisualizationMode, color?: RgbaColor, clipRect?: ScreenRect): boolean;
renderTinted(asset: Asset, rect: ScreenRect, color: RgbaColor, clipRect?: ScreenRect, opacity?: number): boolean;
dispose(): void;
@@ -87,10 +90,11 @@ export function createImageTextureRenderer(context: WebGlRendererContext, invali
}
}
},
render(asset, rect, clipRect, opacity = 1) {
render(asset, rect, clipRect, opacity = 1, rotation = 0, sourceRect) {
const clampedOpacity = clampOpacity(opacity);
if (clampedOpacity <= 0) return true;
const drawRect = clipRect ? intersectScreenRects(rect, clipRect) : rect;
const rotatedRect = rotatedRectBounds(rect, rotation);
const drawRect = clipRect ? intersectScreenRects(rotatedRect, clipRect) : rotatedRect;
if (!drawRect || drawRect.w <= 0 || drawRect.h <= 0) return true;
const entry = getTextureEntry(context, textures, asset, invalidate, () => disposed);
@@ -108,12 +112,12 @@ export function createImageTextureRenderer(context: WebGlRendererContext, invali
gl.uniform1f(opacityLocation, clampedOpacity);
gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
gl.bufferData(gl.ARRAY_BUFFER, rectVertices(context.canvas, rect), gl.DYNAMIC_DRAW);
gl.bufferData(gl.ARRAY_BUFFER, rectVertices(context.canvas, rect, rotation), gl.DYNAMIC_DRAW);
gl.enableVertexAttribArray(positionLocation);
gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);
gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, fullTexCoords(), gl.DYNAMIC_DRAW);
gl.bufferData(gl.ARRAY_BUFFER, textureCoordinatesForCrop(asset.intrinsicSize, sourceRect), gl.DYNAMIC_DRAW);
gl.enableVertexAttribArray(texCoordLocation);
gl.vertexAttribPointer(texCoordLocation, 2, gl.FLOAT, false, 0, 0);
@@ -121,11 +125,13 @@ export function createImageTextureRenderer(context: WebGlRendererContext, invali
gl.disable(gl.BLEND);
return true;
},
renderMasked(asset, rect, maskAsset, maskRect, clipRect, opacity = 1) {
renderMasked(asset, rect, maskAsset, maskRect, clipRect, opacity = 1, rotation = 0, sourceRect) {
const clampedOpacity = clampOpacity(opacity);
if (clampedOpacity <= 0) return true;
const clippedRect = clipRect ? intersectScreenRects(rect, clipRect) : rect;
const drawRect = clippedRect ? intersectScreenRects(clippedRect, maskRect) : undefined;
const contentBounds = rotatedRectBounds(rect, rotation);
const maskBounds = rotatedRectBounds(maskRect, rotation);
const clippedRect = clipRect ? intersectScreenRects(contentBounds, clipRect) : contentBounds;
const drawRect = clippedRect ? intersectScreenRects(clippedRect, maskBounds) : undefined;
if (!drawRect || drawRect.w <= 0 || drawRect.h <= 0) return true;
const entry = getTextureEntry(context, textures, asset, invalidate, () => disposed);
@@ -148,17 +154,17 @@ export function createImageTextureRenderer(context: WebGlRendererContext, invali
gl.uniform1f(maskedOpacityLocation, clampedOpacity);
gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
gl.bufferData(gl.ARRAY_BUFFER, rectVertices(context.canvas, drawRect), gl.DYNAMIC_DRAW);
gl.bufferData(gl.ARRAY_BUFFER, rectVertices(context.canvas, rect, rotation), gl.DYNAMIC_DRAW);
gl.enableVertexAttribArray(maskedPositionLocation);
gl.vertexAttribPointer(maskedPositionLocation, 2, gl.FLOAT, false, 0, 0);
gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, texCoordsForRect(drawRect, rect), gl.DYNAMIC_DRAW);
gl.bufferData(gl.ARRAY_BUFFER, textureCoordinatesForCrop(asset.intrinsicSize, sourceRect), gl.DYNAMIC_DRAW);
gl.enableVertexAttribArray(maskedTexCoordLocation);
gl.vertexAttribPointer(maskedTexCoordLocation, 2, gl.FLOAT, false, 0, 0);
gl.bindBuffer(gl.ARRAY_BUFFER, maskTexCoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, texCoordsForRect(drawRect, maskRect), gl.DYNAMIC_DRAW);
gl.bufferData(gl.ARRAY_BUFFER, textureCoordinatesForRect(rect, maskRect), gl.DYNAMIC_DRAW);
gl.enableVertexAttribArray(maskedMaskTexCoordLocation);
gl.vertexAttribPointer(maskedMaskTexCoordLocation, 2, gl.FLOAT, false, 0, 0);
@@ -166,12 +172,14 @@ export function createImageTextureRenderer(context: WebGlRendererContext, invali
gl.disable(gl.BLEND);
return true;
},
renderMaskRevealPreview(asset, rect, maskAsset, maskRect, opacity, clipRect, layerOpacity = 1) {
renderMaskRevealPreview(asset, rect, maskAsset, maskRect, opacity, clipRect, layerOpacity = 1, rotation = 0, sourceRect) {
const clampedOpacity = clampOpacity(opacity) * clampOpacity(layerOpacity);
if (clampedOpacity <= 0) return true;
const clippedRect = clipRect ? intersectScreenRects(rect, clipRect) : rect;
const drawRect = clippedRect ? intersectScreenRects(clippedRect, maskRect) : undefined;
const contentBounds = rotatedRectBounds(rect, rotation);
const maskBounds = rotatedRectBounds(maskRect, rotation);
const clippedRect = clipRect ? intersectScreenRects(contentBounds, clipRect) : contentBounds;
const drawRect = clippedRect ? intersectScreenRects(clippedRect, maskBounds) : undefined;
if (!drawRect || drawRect.w <= 0 || drawRect.h <= 0) return true;
const entry = getTextureEntry(context, textures, asset, invalidate, () => disposed);
@@ -194,17 +202,17 @@ export function createImageTextureRenderer(context: WebGlRendererContext, invali
gl.uniform1f(maskRevealPreviewOpacityLocation, clampedOpacity);
gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
gl.bufferData(gl.ARRAY_BUFFER, rectVertices(context.canvas, drawRect), gl.DYNAMIC_DRAW);
gl.bufferData(gl.ARRAY_BUFFER, rectVertices(context.canvas, rect, rotation), gl.DYNAMIC_DRAW);
gl.enableVertexAttribArray(maskRevealPreviewPositionLocation);
gl.vertexAttribPointer(maskRevealPreviewPositionLocation, 2, gl.FLOAT, false, 0, 0);
gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, texCoordsForRect(drawRect, rect), gl.DYNAMIC_DRAW);
gl.bufferData(gl.ARRAY_BUFFER, textureCoordinatesForCrop(asset.intrinsicSize, sourceRect), gl.DYNAMIC_DRAW);
gl.enableVertexAttribArray(maskRevealPreviewTexCoordLocation);
gl.vertexAttribPointer(maskRevealPreviewTexCoordLocation, 2, gl.FLOAT, false, 0, 0);
gl.bindBuffer(gl.ARRAY_BUFFER, maskTexCoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, texCoordsForRect(drawRect, maskRect), gl.DYNAMIC_DRAW);
gl.bufferData(gl.ARRAY_BUFFER, textureCoordinatesForRect(rect, maskRect), gl.DYNAMIC_DRAW);
gl.enableVertexAttribArray(maskRevealPreviewMaskTexCoordLocation);
gl.vertexAttribPointer(maskRevealPreviewMaskTexCoordLocation, 2, gl.FLOAT, false, 0, 0);
@@ -338,7 +346,8 @@ function loadTextureState(
const source = asset.source;
image.onload = () => {
if (isDisposed() || entry.source !== source) return;
const texture = createTexture(context.gl, image);
const textureSource = asset.mimeType === "image/svg+xml" ? rasterizeSvgImage(image, asset) : image;
const texture = createTexture(context.gl, textureSource);
if (entry.previousTexture) context.gl.deleteTexture(entry.previousTexture);
entry.previousTexture = undefined;
entry.current = { status: "ready", texture };
@@ -353,6 +362,17 @@ function loadTextureState(
return { status: "loading", image };
}
function rasterizeSvgImage(image: HTMLImageElement, asset: Asset): HTMLCanvasElement {
const canvas = document.createElement("canvas");
canvas.width = Math.max(1, Math.ceil(asset.intrinsicSize.w));
canvas.height = Math.max(1, Math.ceil(asset.intrinsicSize.h));
const context = canvas.getContext("2d");
if (!context) throw new Error("Failed to create SVG rasterization context");
context.clearRect(0, 0, canvas.width, canvas.height);
context.drawImage(image, 0, 0, canvas.width, canvas.height);
return canvas;
}
function renderableTexture(entry: TextureEntry): WebGLTexture | undefined {
if (entry.current.status === "ready") return entry.current.texture;
return entry.previousTexture;
@@ -384,7 +404,7 @@ function intersectScreenRects(a: ScreenRect, b: ScreenRect): ScreenRect | undefi
return { x: x1, y: y1, w: x2 - x1, h: y2 - y1 };
}
function createTexture(gl: WebGL2RenderingContext, image: HTMLImageElement) {
function createTexture(gl: WebGL2RenderingContext, image: TexImageSource) {
const texture = gl.createTexture();
if (!texture) throw new Error("Failed to create image texture");
@@ -422,11 +442,10 @@ function texCoordsForRect(drawRect: ScreenRect, sourceRect: ScreenRect) {
return new Float32Array([x1, y1, x2, y1, x1, y2, x1, y2, x2, y1, x2, y2]);
}
function rectVertices(canvas: HTMLCanvasElement, rect: ScreenRect) {
const x1 = (rect.x / canvas.width) * 2 - 1;
const x2 = ((rect.x + rect.w) / canvas.width) * 2 - 1;
const y1 = 1 - (rect.y / canvas.height) * 2;
const y2 = 1 - ((rect.y + rect.h) / canvas.height) * 2;
return new Float32Array([x1, y1, x2, y1, x1, y2, x1, y2, x2, y1, x2, y2]);
function rectVertices(canvas: HTMLCanvasElement, rect: ScreenRect, rotation = 0) {
const [nw, ne, sw, se] = rotatedRectCorners(rect, rotation).map((point) => ({
x: (point.x / canvas.width) * 2 - 1,
y: 1 - (point.y / canvas.height) * 2,
}));
return new Float32Array([nw!.x, nw!.y, ne!.x, ne!.y, sw!.x, sw!.y, sw!.x, sw!.y, ne!.x, ne!.y, se!.x, se!.y]);
}

Some files were not shown because too many files have changed in this diff Show More