Add on-device background removal to your editor with a single plugin registration — no server uploads, no API keys.

The BackgroundRemovalPlugin adds a “Remove Background” button to the editor dock, and offers one for the inspector bar. The dock button works on the current page’s image. The inspector bar button, which you add with inspectorBarModifier, works on the selected image or page. When tapped, the plugin runs segmentation on-device and replaces the image with the cut-out result. By default it uses IMG.LY’s IS-Net foreground segmentation model — the same model used by the web background-removal plugin — accelerated on the Apple Neural Engine. An alternative Apple Vision backend ships in the same package for portrait-heavy apps that want zero model download.
This guide builds on the Design Editor Starter Kit. The same plugin works with any editor configuration that hosts a dock or an inspector bar.
Install the Plugin#
Add the IMGLYPluginBackgroundRemoval Swift package to your project via Swift Package Manager:
https://github.com/imgly/IMGLYPluginBackgroundRemoval-swiftThe umbrella product re-exports both backends. If you only need one, depend on a sub-library instead to skip the unused dependency:
| Product | Pulls in | Pick when |
|---|---|---|
IMGLYPluginBackgroundRemoval |
Core + ONNX + Vision | You want the default IMG.LY backend, the Vision backend, or both. |
IMGLYPluginBackgroundRemovalONNX |
Core + onnxruntime XCFramework (~20 MB) | You only want the IMG.LY IS-Net backend. |
IMGLYPluginBackgroundRemovalVision |
Core + Vision wrapper | You only want Apple Vision. Skips the onnxruntime binary. |
IMGLYPluginBackgroundRemovalCore |
Core only | You’re implementing a fully custom backend. |
If you depend on a sub-library, import that module instead of IMGLYPluginBackgroundRemoval. The initializers without a configuration: parameter are only available via the umbrella product. Pass configuration: explicitly when you register the plugin.
The default IMG.LY backend downloads its segmentation model from IMG.LY’s CDN when the editor is first presented (default .eager load mode, ~84 MB for the default FP16 model) and caches it on disk. The first user tap is near-instant because the model is fetched and CoreML-compiled in the background while the editor is being presented.
Use the Plugin#
Import IMGLYEditor and the plugin module in the file where you set up the editor:
import IMGLYEditorimport IMGLYEngineimport IMGLYPluginBackgroundRemovalRegister the plugin alongside your editor configuration. Called with no arguments, the convenience initializer uses the default IMG.LY backend — no further setup needed:
Editor(settings) .imgly.configuration { DesignEditorConfiguration() BackgroundRemovalPlugin() }When the user taps a “Remove Background” button, the plugin:
- Reads the image fill of the current page (dock) or of the selected block (inspector bar)
- Runs segmentation on the image bytes
- Composites the foreground over a transparent background
- Replaces the original image, and each resolution in its source set, with the result
The plugin writes each result as a PNG file into the app’s caches directory, and the image refers to that local file. iOS can clear that directory, so persist the results before you save the scene. Upload each file:// URL that engine.editor.findAllMediaURIs() returns, including each resolution of a source set. Then call engine.editor.relocateResource(currentURL:relocatedURL:) with the URL of the upload. The Resources guide covers both calls.
The user can start removals on several images. The model runs on one image at a time, in the order of the taps.
Configure the Plugin#
The following sections show each public customization option.
Pick a Backend#
Pass a BackgroundRemovalConfiguration value to the configuration: parameter to choose the segmentation backend. Omitting it picks the IMG.LY backend with default settings.
VisionBackgroundRemovalConfiguration() selects Apple Vision. This backend has no model download and the smallest binary and memory footprint:
Editor(settings) .imgly.configuration { DesignEditorConfiguration() BackgroundRemovalPlugin(configuration: VisionBackgroundRemovalConfiguration()) }IMGLYBackgroundRemovalConfiguration exposes the IS-Net model’s tuning knobs. Pass an explicit instance to override the defaults:
Editor(settings) .imgly.configuration { DesignEditorConfiguration() BackgroundRemovalPlugin( configuration: IMGLYBackgroundRemovalConfiguration( model: .fp32, modelBaseURL: IMGLYBackgroundRemovalConfiguration.defaultModelBaseURL, loadMode: .lazy, ), ) }The configuration carries three fields, all with sensible defaults:
| Field | Type | Default | Purpose |
|---|---|---|---|
model |
IMGLYBackgroundRemovalConfiguration.Model |
.fp16 |
Which IS-Net model variant to use. |
modelBaseURL |
URL |
IMG.LY’s CDN | Where to fetch the model from. Pass a file:// URL to ship a pre-bundled model. |
loadMode |
IMGLYBackgroundRemovalConfiguration.LoadMode |
.eager |
When to fetch and CoreML-compile the model. |
.fp16produces visually indistinguishable output to.fp32at half the bytes on the wire and runs on the Apple Neural Engine. Pick.fp32only when you have a specific numerical-accuracy requirement..eagerabsorbs the download and compilation cost while the editor is being presented so the first tap is near-instant..lazywaits until the first tap, saving bandwidth and battery if the feature goes unused.
Error Handling#
By default, errors are logged to the console under the subsystem ly.img.plugin.backgroundRemoval. To present a custom error alert, pass an onError closure:
Editor(settings) .imgly.configuration { DesignEditorConfiguration() BackgroundRemovalPlugin(onError: { error in errorMessage = error.localizedDescription }) } .alert("Background Removal Error", isPresented: .constant(errorMessage != nil)) { Button("OK") { errorMessage = nil } } message: { Text(errorMessage ?? "") }Common errors include the Vision backend not finding a person in the image (BackgroundRemovalError.backgroundRemovalFailed), an image that cannot be decoded (BackgroundRemovalError.imageConversionFailed), and IS-Net model-download failures over a flaky network (BackgroundRemovalError.modelDownloadFailed).
Dock Button#
The dock button is prepended by default. Use dockModifier to control where it appears:
Editor(settings) .imgly.configuration { DesignEditorConfiguration() BackgroundRemovalPlugin( dockModifier: { items, button in items.addLast { button } }, ) }The closure receives a Dock.Modifier for the existing items and the plugin’s button; insert the button wherever fits your dock layout. A closure that does not insert the button removes it from the dock.
Inspector Bar Button#
The plugin adds no inspector bar button by default. Pass inspectorBarModifier to add it. This example adds it at the end of the inspector bar:
Editor(settings) .imgly.configuration { DesignEditorConfiguration() BackgroundRemovalPlugin( inspectorBarModifier: { items, button in items.addLast { button } }, ) }The closure receives an InspectorBar.Modifier for the existing items and the plugin’s button; insert the button where it should appear. In the Design Editor Starter Kit, the inspector bar button appears for an image placed on a page, such as one from the Images library, and for a page with an image fill.
API Reference#
| API | Category | Purpose |
|---|---|---|
BackgroundRemovalPlugin(onError:dockModifier:) |
Plugin registration | Default IMG.LY backend, without an inspector bar button. |
BackgroundRemovalPlugin(onError:inspectorBarModifier:dockModifier:) |
Plugin registration | Default IMG.LY backend with the inspector bar button. |
BackgroundRemovalPlugin(configuration:onError:dockModifier:) |
Plugin registration | Explicit backend, without an inspector bar button. |
BackgroundRemovalPlugin(configuration:onError:inspectorBarModifier:dockModifier:) |
Plugin registration | Full initializer: backend, error handler, inspector bar and dock placement. |
BackgroundRemovalConfiguration |
Plugin contract | Protocol every backend configuration adopts. |
IMGLYBackgroundRemovalConfiguration |
Backend configuration | Configures the IS-Net backend (model, base URL, load mode). |
IMGLYBackgroundRemovalConfiguration.Model |
Backend configuration | .fp16 (default) or .fp32. |
IMGLYBackgroundRemovalConfiguration.LoadMode |
Backend configuration | .eager (default) or .lazy. |
VisionBackgroundRemovalConfiguration |
Backend configuration | Configures the Apple Vision backend (people-only). |
BackgroundRemovalProvider |
Plugin extension | Protocol for plugging in a custom segmentation backend. |
BackgroundRemovalError |
Error handling | LocalizedError cases surfaced through onError. |
Next Steps#
- Dock — Customize dock items and ordering.
- Inspector Bar — Customize inspector bar items and ordering.
- Design Editor Starter Kit — The editor preset that hosts this plugin.
- Open the Editor From an Image — Start the editor with image content.