Daniel – IMG.LY Blog

One SDK to Power Them All: Announcing IMG.LY SDK

Eray — Tue, 19 Aug 2025 10:25:44 GMT

From day one, IMG.LY has been driven by a singular mission: to empower creativity at scale. By embedding our technology directly into apps, websites, and workflows, we have helped hundreds of customers from social networks to e-commerce platforms to deliver rich, media-powered experiences to millions of users. We started out by offering two specialized SDKs, the Photo Editor SDK and the Video Editor SDK, later on, we introduced the Creative Editor SDK (CE.SDK) as a more flexible, all-in-one creative editing solution.

As CE.SDK has matured into a foundation for all creative workflows including photo and video editing, we’re taking the next step in our evolution. Later this year, we’ll unify our offerings under a single brand:

IMG.LY SDK.

For existing customers of PE.SDK and VE.SDK, rest assured, both products will continue to receive long-term support. We’ll explain what that means below.

From Fragmentation to Unification

Over the years, our Photo Editor SDK (PE.SDK) and Video Editor SDK (VE.SDK) have enabled customers to deliver creative editing for a specific domain (photo and video). While both excelled within their specific use cases, maintaining separate SDKs brought growing complexity from divergent feature sets across platforms to duplicated codebases and asynchronous upgrade paths. Not only did our customers’ use cases expand, driving demand for more custom editing solutions, this fragmentation ultimately limited our ability to move fast, innovate, and serve this growing demand.

That’s why we introduced Creative Editor SDK (CE.SDK): a unified foundation designed from the ground up to scale across creative domains and workflows. At its core is a single, shared engine capable of handling image, video, layout, text, and audio processing paired with native UI components for every major platform. With each release, CE.SDK has steadily absorbed functionality from both PE.SDK and VE.SDK, making it our most capable and extensible offering yet.

But as this convergence progressed, it also introduced a new kind of complexity: customers were often unsure which SDK to choose. By bringing everything into one consistent, cross-platform solution where every feature is developed in sync for web and mobile, CE.SDK eliminates that uncertainty and simplifies the path forward.

One SDK to Power Them All

Beyond the engine, the SDK now includes prebuilt solutions tailored to common use cases, for example photo, video, or AI-powered design tooling. These solutions help teams get started faster without compromising on customization or scalability.

As we move forward, the IMG.LY SDK will be the sole focus of our platform investments. That means:

Regular updates and new features delivered consistently across all supported platforms.
Turnkey solutions will provide an even simpler integration path, requiring only a few lines of code for set up.
Plugin support and a modular architecture for maximum extensibility.
Automation and Headless Mode for powerful backend or non-UI-driven workflows.
A unified API surface: one learning curve, one integration, fewer surprises.
And for new prospects and existing partners alike, a clear and confident message: IMG.LY SDK is the future.

Long-Term Support for PE. and VE.SDK

As part of this transition, PE.SDK and VE.SDK are now in long-term support mode (LTS). This means:

They remain actively supported by a dedicated team.
They are no longer sold to new customers.
We encourage current users to begin exploring CE.SDK — soon branded as IMG.LY SDK — as the future-ready path forward.

If you’re currently using PE.SDK or VE.SDK, rest assured: we’ve got you covered. But for faster innovation and access to our latest AI-powered features, we strongly recommend evaluating IMG.LY SDK.

What’s Next?

Of course, the journey doesn’t stop here. The unification of our SDKs paves the road for many more exciting capabilities, such as:

Improve Developer Experience
Simpler setup, smarter defaults, and faster time-to-interaction, so teams can go from idea to integration in record time.
Build for Agentic AI
Designed for agent-driven workflows: responsive canvases, multimodal orchestration, and APIs tailored for autonomous actions.
Better Serve Designers & Teams
Our upcoming Studio offering will include project dashboards, version control, and multi-user permissions for collaborative work.
Personalization at Scale
While we already support for variables and templating, we are exploring how to leverage AI to effortlessly create smart variation.
Customizable UI & Fast Workflow Building
We’re making it easier than ever to adapt the editor to your needs whether you’re modifying the UI, embedding your own logic, or orchestrating custom creative workflows.

Now is the perfect time to connect with our team, share your roadmap, test-drive upcoming modules, and plan your migration. Your feedback will help shape the future of IMG.LY SDK.

Vibe-Engineering: When AI Does All the Coding, What Do We Actually Do?

Daniel — Mon, 07 Jul 2025 09:51:05 GMT

This isn’t an article about AI replacing engineers, it’s about discovering what (software) engineering will become when freed from the mechanics of coding itself.

Vibe-Coding, and Vibe-Engineering are omnipresent in my social feeds these days. As with every new trend, it’s hard to judge what really works and what is pure marketing hype. But the central question nagged at me: if AI really can do all the coding, what exactly are we humans supposed to do?

Therefore, I wanted to find out for myself. I didn’t want to build just a Hello World example, so I pulled an old idea out of the closet and dusted it off.

The Experiment: AI Agents Code, Humans Curate

For some time, I had considered porting our IMG.LY background removal library from JavaScript to other platforms using Rust. However, I had postponed this side project due to the anticipated effort involved. This seemed like the perfect project to discover what my role would be in a world where AI does the heavy lifting. To pull it off, I self-imposed only one strict rule:

Hands off the code!

Every commit, every fix, every feature would be handled by an AI coding agent. My role? Being the curator, formulating my intent, reviewing, play-testing, and guiding the agent with feedback.

The agent should be the sole coder, from start to finish.

This was a true hands-off experiment: could an AI build an entire Rust library if I never touched the code?

Is it really something new? After years of handing off development to teammates as a CTO or teaching students, this didn’t feel completely different.

So, what happens when you let an AI do all the coding, and you just review and give feedback? What does it actually mean to be a human in this new paradigm?

The result is baffling and gives a glimpse of what we as engineers can expect for the future.

10,000+ lines of production-ready code. Authored by AI, orchestrated by a human.

Seeing is believing? Then explore the results and judge for yourself at github.com/imgly/background-removal-rs.
I open-sourced the full code and the agent rules used during this experience.

Here’s what I learned: vibe-engineering an entire Rust library with a coding agent CLI as my only pair programmer.

On a quick note, I chose to use Claude Code with Opus 4 and Sonnet 4, as I was already familiar with them and had achieved the best results so far. The costs were kept in check by using a Claude Max Account, which at the time is around ~€100.

The General Workflow That Emerged

After some initial time to get used to working with an AI agent, I realized that the workflow follows pretty closely a typical software engineering flow, but with an important twist that answers our central question: what do humans actually do when AI codes? While most people focus on the code writing part, I discovered that the quality assurance part becomes the human’s primary domain. In my experiments, I realized that to keep the code intact over time, this QA phase is essential—and this is where humans truly shine. Most interestingly, with the right rules and prompting, we can let the agent do the heavy lifting here, too.

Here’s how the collaboration actually worked in practice:

Getting Started
The project began with me setting up the environment and configuring timeouts for long-running tasks—because AI agents need time to compile Rust code and run extensive test suites. Together, the AI and I created a CLAUDE.md file with coding standards and rules (you can see the actual rules we used). We also wrote project requirements documents collaboratively, with me providing the vision and the AI helping structure the technical details.

The Development Dance
Once we got rolling, a natural rhythm emerged. I’d describe what I wanted—something like “add support for different image formats with color profile preservation.” The AI would then create a detailed implementation plan, breaking down the work into manageable chunks. I’d review this plan, often tweaking the approach or scope, then give the green light.

What happened next was fascinating: the AI would write the code, run correctness checks, update all the tests (unit tests, documentation tests, end-to-end tests), run the linter, analyze coverage, and even update documentation and examples. Sometimes it would run performance benchmarks without me asking—the overzealous builder at work.

My New Role: The Quality Guardian
My job became verifying that the intent was met and the API actually made sense. I’d play-test the functionality, try to break it in ways a real user might, and provide feedback. When something worked well, I’d document what we learned for future development. The AI would then write tests to lock in that new functionality, ensuring we never accidentally broke what was working.

This cycle repeated for every feature, with the AI handling the heavy lifting while I focused on direction, quality, and user experience.

The interesting part was that the more I got used to the workflow, the more I handed off to the agent. The validation and general experience had to be largely influenced or orchestrated by me, but building, testing, and even updating documentation could be delegated. In the beginning, I started just building things, but the more we built, the more the AI agent failed to keep existing functionality intact. With those small context windows, it’s kind of understandable after all.

Quality Assurance & Knowledge Capturing Matters More Than Ever

As engineers, we know that at some points we cannot grasp every influence of our code changes on the whole code base anymore. Therefore, we use tests and tooling to help us keep our sanity.

The good thing is that the Rust and Cargo toolchains are exceptionally high quality when it comes to providing the right tooling also for agents.

Due to its lack of long-term context and limited knowledge of all the library’s capabilities, the tests and verification mechanisms become crucial. While these are part of the context, they’re not stored in the agent’s memory but baked into the codebase itself. With the tests and the tooling being available to the agent, it will still make mistakes, as I do, but they can effectively auto-heal or auto-correct their own mistakes with the provided feedback.

I can only advise being even stricter with QA from the very beginning.

Another thing that became apparent very quickly is that, unlike a human colleague with lots of context and a good memory, agents still need significant help to remember things to avoid rediscovering the same information repeatedly. This revealed another crucial human role: context engineering becomes critically important. As such, we engineers become the keepers of institutional knowledge, the ones that help the agent remember what worked and what didn’t. I would describe it as knowledge capture, for lack of a better term.

Tips & Tricks

AI Is Eager, Resourceful, and Has Memory Like a Sieve

Here are the key insights I gained from building the library with an AI coding agent:

Code that works: This might seem obvious, but it wasn’t clear to me if the library would ever be usable or publishable, but it is. The AI proved it could deliver production-ready code.

Time Effectiveness: While creating the library took around 3 weeks, don’t let this fool you. Most of the time, the agent worked alone, and I only had to check in once in a while to review and give feedback. Iteration cycles and rewriting still take time. So I wouldn’t say development was faster than if I did it myself, but much more scalable and effective with my own time. Here’s what I actually spent my time on: reviewing architectural decisions, ensuring the API made sense, and play-testing the final product.

Stamina: The agent never complained, even when fixing 400+ warnings after setting the linter to pedantic—it pushed through. The AI powered through hundreds of warnings, even clustering them and suggesting which to fix first. To be fair, most humans would have either gone crazy or aborted the effort claiming “it’s good enough.” This revealed something important: as humans, we don’t need to be the ones grinding through tedious tasks anymore.

Overzealous Builder: The agent often built more than I asked for. Scoping and clear implementation plans were essential. This taught me that one of the key human roles is setting boundaries and maintaining focus.

Premature “Done”: Sometimes the agent claimed it was finished but left code mocked or incomplete. I learned to always ask, “Is there anything left to implement?” Quality assurance becomes fundamentally a human responsibility.

Old Knowledge: The AI sometimes relied on outdated library knowledge. I had to remind it to check the latest docs and version of dependencies, but when asked, it searched the web, GitHub, crates.io, etc., and gathered all necessary information for using a library. Humans become the validators of information freshness.

Forgetting and Compaction: Larger features grow outside the context window, so the agent would “compact” context, sometimes forgetting important details. Forcing it to write implementation plans in markdown files helps maintain flow and allows better resuming and bookkeeping. We humans become the keepers of long-term project memory.

Helps Establishing Rules: While working with the agent, I saw repeating patterns like it forgetting how to check, build, and test the application, or best practices like using git worktrees. I started adding new rules to Claude.md by myself at first, but quickly realized that the agent is far better at formulating and adding new rules if I asked it to. The human role evolved into being the pattern recognizer and rule creator.

Doesn’t Always Obey the Rules: Most of the time, the rules are followed, but occasionally it doesn’t follow them and forgets to automatically run all tests. This reinforced that humans must remain the final guardians of process and quality.

Code Deletion: Occasionally, the AI “fixed” issues by removing important code paths. Solution: Insist on never removing functionality without discussion, and build a robust test suite. We become the protectors of existing functionality.

Process Management: The AI struggled with background processes, understanding when to run things in the background, and keeping track of what it started. For example, starting a web server via Bash (see Claude Code Bash tool docs), and then trying to use a playwright tool (see also playwright-mcp) to access this server was blocked by these issues. Complex orchestration remains a distinctly human skill.

Overly Agreeable: Too often, it seems to just agree with whatever I said. For the future, I wish agents wouldn’t be so obedient. This highlighted that humans need to be the challengers and devil’s advocates in the process.

Further Remarks

After three weeks of this experiment, the patterns became clear. Here’s what I learned about the human role in AI-assisted development:

Ensure Verify, and Self-Correct: Always ask the AI to check, format, lint, test, and benchmark its work. Your job becomes quality orchestration, not quality execution.

Always Plan First: Insist on a clear implementation plan before coding begins. Humans excel at high-level architectural thinking and breaking down complex problems.

Write It Down: Let an agent keep notes, todos, and open issues in markdown files. Ideally in the repository itself. Don’t rely on the AI’s memory. We become the institutional memory keepers.

Scope Features: Break tasks into small, manageable pieces. Feature scoping and boundary setting become a core human skill.

Test Suite is King: A robust test suite prevents accidental “fixes” that remove functionality. Humans become the guardians of existing behavior.

Stay Involved: Fast iteration and discussion with the AI is crucial. Don’t go fully hands-off. Active curation and guidance remain essential.

Conclusion

So, what do humans actually do when AI does all the coding?

To answer that, let’s look at what AI agents are today:

Agents are super-talented, overly obedient coding assistants with lots of stamina and endless potential.

They’re fast, never complain, and iterate like champs. But they need your guidance, structure, and a healthy dose of skepticism.

In the end, the best results come from a true partnership: you plan and guide—the AI builds, fixes, and learns.

If you follow these concepts, then

“Vibe engineering is a highly engaging experience.”

And if you’re wondering: yes, I’d do it again. But next time, I’ll make sure the AI writes everything down too, and I’ll put even more effort into validating and securing new features as soon as they’re play-tested and verified. The human role isn’t disappearing—it’s evolving into something more strategic and impactful.

Through this experiment, I’ve identified three fundamental shifts that define what engineering becomes in an AI-assisted world:

From Execution to Orchestration
We’re no longer the ones typing code—we’re the conductors. Our new skills revolve around prompting (framing tasks so AI understands our intent), directing workflows (knowing when to use AI versus human judgment), and tool fluency (combining different AI tools to create something greater than their parts).

From Knowledge to Reasoning
AI can recall facts and syntax better than any human ever could. But what matters now is our ability to interpret, contextualize, and make sense of uncertainty—something current AI still struggles with. We become the ones who ask “why” and “what if” rather than just “how.”

From Repetition to Adaptation
Skills rooted in routine are increasingly automated. The enduring value lies in adaptive thinking, problem-framing, and reinventing approaches when the usual patterns don’t apply. We’re the ones who recognize when something feels off, even if we can’t immediately articulate why.

Afterthought

During this experiment, I saw that the agent repeated similar tasks over and over again. One thing that struck me was looking up documentation. Whenever something with a third-party library didn’t pan out, it started web-searching for info and guides about the library. It rarely started out with reading the Rust docs first, so I had to tell it to use the Rust docs. Therefore, I assume that providing quick access via tools to guides, docs, API references, and examples will spare me some roundtrips, time, and tokens and accelerate the process. I know that there are tools like context7, but it seems to be focused on JavaScript. At least for the Rust ecosystem, the docs are centralized and standardized and can even be used locally. A tool to access those quickly and find things in them would be highly beneficial.

Additionally, for coding projects, there are typical steps to follow to verify if the code is sound. It’s probably also best to provide specific tools or workflows to follow these steps exactly. Rules already help with this, though.

Last but not least, a natural next step would be for an agent to analyze its history periodically and provide new memory entries (rules) based on that to reduce repetition and common mistakes. For more details, see the Appendix: Most used prompts section, where I analyzed the top 10 most common prompts to help agents proactively develop useful rules.

If you’re curious about improvements, I’ve put together a wishlist for Claude Code that would make the agent even more useful.

Try It Yourself

If you want to experience the results of vibe-engineering firsthand:

# Clone and explore the codebase
git clone https://github.com/imgly/background-removal-rs.git
git checkout --tag v0.2.0

# Or install the CLI directly
cargo install --git https://github.com/imgly/background-removal-rs.git --tag v0.2.0

Remember: every line of code in this library was written by AI while I played the role of curator, architect, and quality guardian. Judge for yourself whether this new paradigm produces production-ready results.

Appendix

Appendix: Claude Code Wishlist

During the experiment, I encountered some issues whose resolution would improve the agent.

Project-scoped history (not global).
Project-scoped todos (not global).
Project-scoped implementation plans (not global).
Customizable /compact prompt or choose other compact strategies.
Improved handling of background processes.
History analytics with memory proposal.
Code-specific predefined Code and Validate workflows.
Allow forking of multiple agents from a single point in the session history to create multiple trials with the same intent and context.

Appendix: Most Used Prompts

Claude code stores the history under ~/.claude/history as json formats. I asked claude code to read them in and categorize the most used prompts.

#	Category	% Usage (Count)	Description / Examples
1	Other/Specific Instructions	19.5%	Technical specs, detailed requirements
e.g.: “Relax the success metrics”, “Use tensor data directly as alpha channel”
2	Questions	12.7%	Status checks, clarifications
e.g.: “What’s next?”, “How can I test it myself?”, “Do we have any mock implementations?“
3	Implementation Requests	11.1%	Direct build/create requests
e.g.: “Create a PRD for Rust port”, “Make ONNX Runtime injectable”
4	Simple Responses	10.0%	Short confirmations, approvals
e.g.: “ok”, “go for it”, specific technical choices
5	Continuation Commands	8.4%	Requests to proceed
e.g.: “go on” (59 times)
6	File Operations	8.3%	File management
e.g.: “Write this into PRD.md”, “Move packages into crates directory”
7	Context Summaries	8.3%	Session continuation due to context limits
e.g.: “This session is being continued from a previous conversation…“
8	Analysis/Review Requests	7.7%	Requests to analyze/review
e.g.: “Analyze my background removal project”, “Check the preprocessing”
9	Bug Fix/Issue Resolution	4.4%	Problem identification/fixes
e.g.: “You mix up release and debug”, “This is wrong”
10	Testing	3.8%	Running/validating tests
e.g.: “Test images are incorrect”, “Rerun comparison tests”

Appendix: Claude Code Settings

In real-world projects, the default timeout of two minutes is not enough. I bumped the timeouts to the maximum values to allow execution of unit tests, E2E tests, benchmarks, and long-running tasks.

BASH_DEFAULT_TIMEOUT_MS: Sets the default timeout (in milliseconds) for long-running bash commands.
BASH_MAX_TIMEOUT_MS: Specifies the maximum timeout (in milliseconds) that can be set for bash commands.
BASH_MAX_OUTPUT_LENGTH: Limits the maximum number of characters in bash outputs before they are truncated in the middle.
CLAUDE_BASH_MAINTAIN_PROJECT_WORKING_DIR: Ensures the working directory is reset to the original after each Bash command.

{
  "env": {
    "CLAUDE_BASH_MAINTAIN_PROJECT_WORKING_DIR": "true",
    "BASH_MAX_TIMEOUT_MS": "3600000",
    "BASH_DEFAULT_TIMEOUT_MS": "3600000",
    "MCP_TIMEOUT": "3600000",
    "MCP_TOOL_TIMEOUT": "3600000"
  }
}

Library Capabilities

High-performance Rust library for AI-powered background removal with hardware acceleration, built for
production scale and developer productivity.

Performance Highlights

Hardware acceleration: CUDA (NVIDIA), CoreML (Apple Silicon), CPU fallback
Sub-second processing on modern hardware (100-1200ms depending on image size)
Memory efficient with optimized threading and session reuse

Key Features

AI Models & Quality

Multiple state-of-the-art models (ISNet, BiRefNet)
FP16/FP32 precision variants for performance vs. quality
Portrait-optimized and general-purpose models
Custom model support via ONNX models from Hugging Face

Format and Color Profile Support

Input: JPEG, PNG, WebP, TIFF, BMP with ICC color profile preservation
Output: PNG (transparency), JPEG, WebP, TIFF, raw RGBA8

Integration Patterns

One-liner API for simple use cases
Session-based processing for batch operations
Stream processing from any AsyncRead source
CLI tool for standalone usage and pipeline integration

Architecture & Platforms

Dual Backend System

ONNX Runtime: Maximum performance, GPU acceleration
Tract: Pure Rust, zero external dependencies

Platform Support

macOS: Apple Silicon + Intel with CoreML acceleration
Linux/Windows: NVIDIA CUDA + CPU fallback

Developer Experience

Modern Rust Ecosystem

Async/await native support
Comprehensive documentation and examples
Zero-warning policy with extensive testing
Structured tracing for production observability

CLI Capabilities

Batch processing with recursive directory support
Model management (download, cache, clear)
Provider diagnostics and performance monitoring
Pipeline-friendly with stdin/stdout support

3,000+ creative professionals get early access to new features and updates—don’t miss out, and subscribe to our newsletter.

Background Removal in the Browser Using ONNX Runtime with WebGPU

Daniel — Tue, 11 Jun 2024 16:26:17 GMT

TL;DR: Using ONNX Runtime with WebGPU and WebAssembly leads to 20x speedup over multi-threaded and 550x speedup over single-threaded CPU performance. Thus achieving interactive speeds for state-of-the-art background removal directly in the browser.

Removing background from an image is a typical job to be done in creative editing. We have come a long way from manually knocking out the background from an image to full automation with Neural Networks.

Most state-of-the-art background removal solutions work by offloading the task to the server with a GPU as it was simply infeasible to run the NN on the client.

However, running background removal directly in the browser offers several advantages over server-side processing:

Reduced server load and infrastructure costs by offloading heavy lifting to the client.
Enhanced scalability by distributing the workload across client devices.
Easier compliance with data protection and security policies by not transferring data across a network to a server.
Offline processing without needing a reliable internet connection.

It caters to a wide range of use cases, including but not limited to:

E-commerce applications that need to remove backgrounds from product images in real time.
Image editing applications that require background removal capabilities for enhancing user experience.
Web-based graphic design tools that aim to simplify the creative process with in-browser background removal.

In general, two factors influence the feasibility of running background removal directly on the client.

The execution performance, and
the download size of the Neural Network.

The performance or overall runtime is the major factor to be useful in interactive applications, if a user has to wait several minutes or hours for a neural network to execute, this is in many cases far too long in terms of good user experience. From experience, there are three factors to consider.

The initial first-time execution. The major factor is that neural networks come with the drawback of generally being several MB to GB in size, thus the time to download the neural network into the browser cache is considerable. In subsequent browser page reloads this has no impact anymore.
The neural network or session initialization time, cannot be cached and has to run with every reload of the page in the browser.
The neural network or session inference time, largely depends on the longest path inside the neural network and most importantly the execution time of each operator in the neural network.

Towards Real-time Background Removal in the Browser

Neural networks are commonly trained in frameworks like PyTorch, which is a neural network library for Python, as such not usable directly in the browser. The best option to run neural networks directly in the browser is converting the neural network into the Open Neural Network Exchange (ONNX) format, which is a widely supported standardized format by Microsoft used extensively in the industry.

These ONNX-formatted neural networks can then be reconverted into a platform-specific format or directly executed by a supported runtime.

The ONNX Runtime by Microsoft is a high-performance inference engine designed to run ONNX models across various platforms and languages. One notable feature is ONNX Runtime Web, which allows JavaScript developers to execute ONNX models directly in the browser. ONNX Runtime Web offers several execution providers for hardware acceleration. For instance, its WebAssembly execution provider enhances CPU execution performance using multiple Web Workers and SIMD instructions.

More importantly, starting from version 1.7.0, ONNX Runtime Web includes official support for the WebGPU execution provider. WebGPU is a modern web API that enables developers to utilize GPU power for high-performance computations, offering a significant performance boost over CPU-based in-browser machine learning. WebGPU support has been available by default since Chrome 113 and Edge 113 on Mac, Windows, and ChromeOS, and Chrome 121 on Android. For the latest browser support updates, you can track them here.

Implementation Details

For the implementation of the open-source package @imgly/background-removal-js, we started the journey with a neural network implementation in PyTorch written in Python. The network was then converted to ONNX. You can see our live showcase here.

The original model was using 32-bit floating point (fp32) precision, which is fine, but results in a file size of 168 MB after converting it to ONNX. As mentioned earlier, the size of the network has a large impact on perceived first-time execution performance as the download time tends to be longer than the execution time, but more to that later.

To reduce the size of the model, we converted the model to use fp16 (16-bit floating point) and QUINT8 (Quantized 8-bit) datatypes. Thus, effectively reducing the size to half (84MB) and a fourth (42MB) of the original size. Additionally to the download size, the operators used will be converted corresponding to the datatype and different data types and depending on the hardware may lead to speed improvements or even deteriorating due to specialized hardware being used or not being present.

Note, that while that sounds great, the conversion has a potential negative impact on the quality of the output as we are removing information in the neural networks and since we are working with images, artifacts might become visible and the quality of the resulting background mask is reduced.

Evaluation

When dealing with neural networks in the browser we can identify three different scenarios

First Use
First Run
Consecutive Runs

The first-use scenario occurs when the web application is started the first time. The neural network has to be downloaded from the server into the browser sandbox, as neural networks are several 10-1000 MB in size this is not neglectable.

The first run assumes that the neural network is already present in the browser cache, however, when the neural network ought to be used, the network has to initialize before execution.

In consecutive runs, the neural network is already in memory and the execution time is largely determined by the neural network depth and operators only.

First Use – Neural Network Download

While download or time is a large factor in the first-time execution time, this is largely dependent on the available network bandwidth, as such it is not part of the evaluation but in order to get an idea here are the expected download times for the fp32 (168MB) and fp16 (84MB) neural network subject to various common network bandwidth:

Network Bandwidth	Filesize	Download Time
10 Mbps	84 MB	67s s
100 Mbps	84 MB	6.7 s
1 Gbps	84 MB	0.67 s
10 Gbps	84 MB	0.067

Network Bandwidth	Filesize	Download Time
10 Mbps	168 MB	134.4 s
100 Mbps	168 MB	13.4 s
1 Gbps	168 MB	1.34 s
10 Gbps	168 MB	0.13 s

Based on the assumption that the median download bandwidth is ~100 Mbps, we see that it’s in the 5-15 second range.

First Run – Neural Network Initialization / Compilation

Before the neural network can be executed it has to be initialized. Initialization includes several execution provider-specific steps. Most prominent are the time to upload or convert the data to the execution provider and execution provider-specific ONNX graph optimization passes. This all adds to the first run experience. We have evaluated the average session initialization time for the CPU (WASM) and WebGPU provider on a MacBook Pro 13” from 2024 with an Apple M3 Max 16 cores to get an idea of the general impact on first run execution time:

Device	Datatype	Session Initialization Time
CPU (WASM)	fp32	~320ms
CPU (WASM)	fp16	~320ms
WebGPU	fp32	~400ms
WebGPU	fp16	~200ms

Note, that session initialization is not negligible and adds significant runtime overhead and might be subject to additional optimization possibilities like caching the optimized model.

Consecutive Runs – Neural Network Execution

Independent of the download and initialization time, we evaluated different execution providers on a MacBook Pro 13” from 2024 with an Apple M3 Max 16 cores. While this is top-end consumer hardware, the general trends will probably apply to various hardware configurations. As a reference, we choose the single thread performance with the neural network running on the CPU with 1 worker thread and SIMD disabled.

Device	SIMD	Threads	Datatype	Session Runtime	Speedup
CPU (WASM)	No	1	fp32	~53000 ms	1.0 x
CPU (WASM)	No	16	fp32	~6300 ms	8.4 x
CPU (WASM)	Yes	1	fp32	~15000 ms	3.5 x
CPU (WASM)	Yes	16	fp32	~2000 ms	26.5 x

The data above reveals that running the neural network without any acceleration such as SIMD and threading in the browser results in almost 53s runtime, and as such is for most interactive use cases too slow to use. Due to the optimizations of the ONNX Runtime that uses SIMD and threads, we can achieve an overall speedup of roughly ~26 times compared to the baseline performance. Thus decreasing the session runtime to around 2s and making it usable for interactive applications.

As mentioned before, we neglected the download time and size of the network. Leveraging the fp16 model compression, we re-ran the benchmarks and got similar results as before, but with half the bandwidth required to download the network for the first time application. The general visual performance of the fp16 and fp32 is similar or visually not perceivable. The quint8 model led to artifacts and as such unusable for visual processing, thus we excluded from the following benchmark.

Device	SIMD	Threads	Datatype	Session Runtime	Speedup
CPU (WASM)	No	1	fp16	~55000 ms	1.0 x
CPU (WASM)	No	16	fp16	~7300 ms	7.2 x
CPU (WASM)	Yes	8	fp16	~15000 ms	3.5 x
CPU (WASM)	Yes	16	fp16	~2300 ms	23.9 x

The results are a little worse than the fp32 version, which might be because the fp16 datatype has no direct specialized hardware in modern CPUs like the M3 Max CPU, and as such additional fp16 to fp32 conversion operations have to be performed.

Our final benchmark measures the WebGPU performance, which led to the following impressive results. To have a fair comparison and understanding of the impact of the WebGPU technology, we compare it with the best CPU version with 16 threads and SIMD enabled with the fp32 model, which was around ~2 s.

Device	SIMD	Threads	Datatype	Session Runtime	Speedup
WebGPU	Not applicable	Not applicable	fp32	~120ms	16.6 x
WebGPU	Not applicable	Not applicable	fp16	~100ms	20.0 x

The WebGPU performance varies around 16 to 20 x improvements over the best CPU execution time. The GPU has specialized hardware for fp16 instructions, as such the model performs better than the fp32 model. Also, note that the session initialization time is also reduced due to half the required bandwidth to upload the network data to the GPU.

Therefore, the first run of the network will take ~300 ms and consecutive runs will be ~100 ms, leading to near real-time performance in the browser.

Note, that the WebGPU performance is an astonishing 550 times faster than the single thread, with no SIMD performance.

Try the live implementation in our background removal showcase.

Conclusion

WebGPU is a major leap in establishing the browser as a factor to be reckoned with as a true Application platform. With IMG.LY, we are striving to leverage modern technology to make design tools accessible, this includes on-device execution, on-premise, but also on-cloud execution of design tools leveraging neural networks.

As a next step, we will port background removal to all our supported platforms, ONNX Runtime seems the best choice, as it is already available for all the potential platforms we support.

Furthermore, we are evaluating the feasibility of in-browser background removal for videos to be included with our video-editing suits.

Try our tools today to see how we help bring unique creative editing experiences to any application. Or contact us to discuss your project.

Unleash Creativity with CE.SDK’s New Plugin System

Daniel — Mon, 03 Jun 2024 08:42:25 GMT

At IMG.LY, we have always believed that a superb design editor should be effortlessly customizable and extensible. We are thrilled to roll out a brand-new Plugin system for CE.SDK in the upcoming months—to take creative editing and feature development to the next level.

Starting with one-click features like background removal or vectorizers, smart design tools like QR codes or subtitle generators, and deeply interactive features like generative AI for text and images; all these tools can be used or built by customers soon.

Additionally, our upcoming Plugin system will bring you unparalleled autonomy by making feature development for our SDK accessible and offering extensive options to reconfigure our editor’s UI.

Start exploring our Plugin System rollout now to immediately benefit from upcoming features.

Built for Modification

Let’s dive deep into some of the opportunities CE.SDK plugins help unlock.

Unlocking the AI Revolution

While the AI transformation is already fully underway, much of the tech is still not very accessible to product builders, often requiring deep technical knowledge to get started. At the same time, AI features become significantly more valuable when integrated with other editing functionalities in workflows or automation. With our plugins, we aim to make it effortless to leverage innovation and put it to use.

Boosting Customer Autonomy

Key to our success is providing maximum flexibility and autonomy to our customers about product decisions. Ultimately, you shouldn’t depend on our product roadmap; rather, you should be able to add features when you like. While our SDKs are already highly configurable, plugins allow tailoring the whole user experience not only on a look & feel level but through custom functionalities and editing experiences.

Accelerate Product Expansion

Many ecosystems witnessed explosive growth in added value to the user after releasing plugin mechanisms. Currently, only IMG.LY core developers can contribute to the SDK. We have started to extend this to solution engineers and even designers on our team who don’t have much knowledge of the inner workings of the SDK. Ultimately, we will push this more and more into a community of contributors, making the community’s innovation accessible to everyone.

Key Concepts

Three important concepts have driven the development of our Plugins:

Customizable Menu Bars
We are extending our API so that it allows easy hooking into various parts of the UI. Our editor has key components like the inspector, toolbar, and on-canvas menu. These are now all accessible through an API, so you can hook your feature anywhere in the editor.

UI Building Blocks to Provide Consistency
To reach a high level of consistency and speed up time for development, we will be providing out-of-the-box UI components such as buttons, sliders, text inputs, etc.

Escape Hatches
From experience, we know that sometimes unique functionality needs unique solutions, so we have added escape hatches to add custom elements via HTML whenever needed.

What Can You Build with our Plugin System?

Let’s explore some potential use cases of the plugin system.

Custom Actions

Adding custom actions is a great option to make simple third-party APIs accessible within the editor. This can be one-click edits such as background removal, vectorizers, or auto-enhancement for images. You can also add custom actions for text in combination with Large Language Models (LLM) to provide features like autocorrection and improved writing, etc.

Custom Tools

Some custom functionality will require more than just a single button, e.g., to generate AI images, background patterns, QR codes, or maps. In these cases, you’ll require sliders, text input, drop-downs, and many other UI elements. With plugins, you can easily create panels with your own UI to bring any custom tool to life.

Custom Assets & Presets

Apart from building custom tools, you can also bundle and group effects into presets and make them accessible in a custom panel. This is especially useful to simplify the design process or create standard design components: for example, providing beautiful text presets will enable your users to create instantly great text designs without any design knowledge.

Custom Libraries

Integrate third-party libraries, such as Unsplash, Getty Images, Pexels, or your own.

Custom Editor Behavior

Some of our customers asked us how they could move a functionality from one place in the editor to another. Let’s say you wish to move the function ‘move to front’ from the inspector to the canvas menu. This is not a problem! You can do this by using the internal API endpoints of our editor.

As for custom editor behavior, you can do far more than just move functionality. Here is a demo of layer lists we built as a plugin.

Custom User Feedback

Additionally, you can enhance the canvas with overlays, which are useful for providing alerts, instructions, or feedback directly on the canvas.

What’s Next

We are now rolling out Plugins and building an initial set of features through Plugins ourselves—available for the web first, and mobile SDKs will follow. Keep your eyes peeled for our next releases and don’t hesitate to get in touch with us to learn more about plugins, and how your product benefits from integration without losing time and resources.

Thank you for reading. Join 3,000+ creative professionals—subscribe to our newsletter for updates on new features, plugins, early access, and more!

CE.SDK v1.7.0 Release and Roadmap

Malte — Mon, 08 Aug 2022 12:07:47 GMT

Today, we are thrilled to highlight the best new features and changes in CE.SDK since our v1.6.0 Release. We have also published our Product Roadmap for CE.SDK to share our vision and direction for upcoming releases. This roadmap will let you know ahead of time what features you can expect from us and accelerate your product planning and development. We are excited to hear your feedback and suggestions to build more powerful features for your application!

CE.SDK v1.7.0

This release includes:

Auto Close Option in Web UI Configuration for Asset Libraries
Custom URI Resolver
Improving Mobile Touch Support
Rotation of Groups and Hierarchies
Custom Asset Source UI
Disconnecting UI Variants from Roles
Asset Source ApplyAsset API
Blur API
Effects API
Image Fills API
Image Straighten API
Improved Group Selection and API
Scopes API
Shadows API
Zoom API

Auto Close Option in Web UI Configuration for Asset Libraries

There are no one-size-fits-all solutions for the behavior of asset library panels after an action was triggered. It depends on the use case and might vary between the panels and the available screen space. Thus, this is now configurable.

Custom URI Resolver

Custom URI Resolvers give you full control over how URIs should be resolved. You can create custom storage backends and even resolve different resolution images in different instances. Find more information in our documentation.

Improving Mobile Touch Support

Input handling was streamlined to handle both touch and mouse input efficiently. On touch devices, all handles maintain the recommended minimum size for touch input and are intelligently hidden to minimize overlap. Thanks to this mechanism, resizing and rotating are always a breeze.

Rotation of Groups and Hierarchies

The engine and user interface add options for rotating groups and design blocks with children. This allows changing the rotation of multiple elements at once quickly.

Custom Asset Source UI

Instead of fixed entry points such as “Images,” “Stickers,” or “Shapes,” we want to enable custom and flexible entry points. Until today, we tied them to a block type, which is currently only relevant when users add an asset to the scene.

Moving from Roles to Abilities: Disconnecting UI Variants from Roles

Before the change, the Advanced UI was tied to the creator role and the Default UI was tied to the Adopter Role. This is now no longer the case. The scopes and editing options are still controlled by the abilities of the current role.

Shadows in Advanced UI

With this update, we roll out the long-awaited ability to apply shadows to elements in our Advanced UI. Realistic drop shadow effects add more depth to your designs with a few clicks – no other editor is needed.

Blur API

We introduce new APIs to modify the blur for images and other blocks. This change allows programmatic access to add blur effects in automation cases and when building your own UI.

Effects API

We introduce new APIs to modify the effect stack for shapes and other blocks. This allows programmatic access to effects like adjustments, LUT filters, and many more.

Image Fills API

Currently, you can only access and use images with the image block. However, pages or vector shapes could use more than just a color or gradient fill. Therefore, we broaden the concept of fills by allowing every block to have an image fill, e.g., pages. This change will enable building a more natural photo- or video-editor experience. Documentation.

Image Straighten API

We are extending the Image Crop API to allow straightening, flipping, and rotating images during cropping. This change allows more fine-grained control.

Improved Group Selection and API

There are multiple options to select blocks inside a group:

selecting a group first and with each click select one block deeper in the hierarchy
selecting an element under the cursor directly, neglecting the hierarchy

Previously, we defaulted to different behaviors depending on the user’s role. Now, this is configurable and also customizable during runtime via APIs. Therefore, it’s up to the implementor how the selection should work.

Scopes API

We introduce new APIs to modify the scopes for blocks. This update allows programmatic access to change the scopes for users and allows control over the possible edits a user can make to a scene.

Shadows API

We introduce new APIs to modify shadows for images and other blocks. This change allows programmatic access to add shadow effects to be used in automation cases and when you build your UI.

Zoom API

We introduce new APIs to modify the current zoom level. This allows programmatic access to change the current zoom easily for automation use-cases or for building your own user interface.

Product Roadmap

As mentioned before, we are happy to publish our Product Roadmap to let you in on our new features and enhancements for CE.SDK. This way, we can elaborate our decisions and plans for the product and how you can benefit while allowing you to give us feedback and indicate the importance of particular features.
Now that you know what’s coming down the pike, you can prepare for the release of significant improvements and new features such as Video Support for Web or Native iOS Support well in advance.

CE.SDK v1.6.0 Release

Malte — Fri, 20 May 2022 09:21:48 GMT

Today, we are highlighting the best new features and changes in CE.SDK since our v1.5.0 Release six weeks ago. CE.SDK v1.6.0 is adding:

Linear Gradient Fills
Strokes, Outline, Insets, and Stroke Patterns
Blend Modes
Scale Groups, Multi-Selection, and Hierarchies
UI Template Library
Floating and Static Asset Library Panels
Placeholders API
Editing State APIs
API for Image Fit Modes

Linear Gradient Fills

With this update, we are introducing linear gradient fills. This feature creates a gradual blend between two colors, including transparency. Use the Color Picker to switch between solid color and linear gradient fill modes and determine the order of colors.

Strokes, Outline, Insets, and Stroke Patterns

This feature introduces strokes and stroke patterns. Strokes apply to any block, including images, text, and shapes. Use our APIs or UIs to set the stroke width, color, style, and more options affecting the stroke appearance.

Blend Modes

Blend modes are a popular feature in creative and image editing tools. They blend two blocks or layers to get different types of effects. This feature is available in both the API and the UI.

Scale Groups, Multi-Selection, and Hierarchies

Our engine and user interface now provide options to scale groups, multi-select and create design blocks with children. You can easily change the scaling of multiple elements at once. This also simplifies resizing and rescaling your template designs for other page and art board sizes.

UI Template Library

We are introducing the template library in all UIs. Allow users to switch between multiple templates while editing.

Floating and Static Asset Library Panels

For some screen sizes and use-cases, it might be tedious that the asset library floats over your canvas and covers your design. Therefore, you can now configure the asset library panel to be either floating or static. A static asset library won’t overlap with your design and adjusts to your screen size perfectly.

Placeholders API

CE.SDK 1.6 introduces an additional API that allows you to interact with all placeholders on the canvas. The API queries the number of placeholders in a template or document and lists all available placeholders in the current scene.

Editing State APIs

We extend the API to allow querying and modifying the current editing mode. That includes changing the on-canvas editing mode to Crop, Text, or others. You can also query the editor state for helpful information like the desired cursor type and rotation and the text cursor position. To observe these values more comfortably, an additional endpoint allows registering for callbacks whenever the editing state changes.

API for Image Fit Modes

The API now allows switching between fit modes for images:

Crop applies crop properties, and ignores the aspect ratio.
Cover resizes the image aspect-aware to fill the frame.
Contain allows resizing the image aspect-aware to fit into its frame.

CE.SDK v1.5.0 Release

Daniel — Thu, 14 Apr 2022 12:53:45 GMT

Another six weeks have passed since our v1.4.0 release that brought an updated asset library and headless APIs. Since the initial CE.SDK release, we’ve been listening to your feedback and continuously releasing updates to improve and expand CE.SDK to better serve your needs. Here’s a short overview of what’s happened in the last quarter.

CE.SDK v1.5.0 is adding:

Hierarchies (Groups)
Template Imports & Exports
More Performance, More Power
High-Performance Rendering
Improved Touch Support
PDF Export
Subscription/Events API

Hierarchies (Groups)

With this release, we finally extended our flat hierarchy model to the typical parent-child relationship model between items on the canvas. That allows you to build an even more complex design: Group items to form a single unit and handle those items as one or control their opacity.

Template Imports & Exports

Export your template scenes as .scene-files that pack all assets that were used. You can also share your template by URL - all dependencies will resolve accordingly.

More Performance

We have replaced our underlying rendering engine with a new high-performance implementation, resulting in the performance of distinct scenarios improving by 100%. Likewise, as a result of polishing up various logic, CE.SDK runs faster than ever.

Improved Touch & Mobile Support

After evaluation, CE.SDK v1.5.0 expands upon our mobile advancement efforts by improving our web UI for touch inputs on mobile devices.

PDF Export

CE.SDK now supports PDF export. Vector paths and images are embedded, and rasterization will only be applied when necessary. Scenes of multiple pages can be exported as a multipage document.

Build Your Custom UI

We vastly improved support for building custom UIs on top of CE.SDK engine:

The CreativeEngine offers a powerful set of APIs to inspect, modify and interact with a scene.
A new set of UI-specific APIs, including history and zoom management, was added as a preview.

Events API

An Event API allows subscribing to scene changes and updates custom UI components accordingly.

Other

For an even smoother UI experience, we have improved the visual quality of element state indicators, and they now respect the element order.

CE.SDK v1.4.0 Release

Daniel — Tue, 22 Feb 2022 12:34:56 GMT

Six weeks have passed since our last release, where we shipped our new and improved default UI, Placeholders, and a preview for our new Dashboard.

This time, we focused heavily on making content discovery and provisioning better for you and your customers, as well as automation and API-driven design. Our release includes:

Custom Asset Libraries & Management
Internationalization
Headless Mode & APIs
Other
Outlook on upcoming releases

Asset Libraries & Management

With this release, we improve our library system and overall usability. CE.SDK 1.4.0 introduces a new toolbar to let your users quickly access their favorite images, shapes, stickers, or text assets.

Even more importantly, we added a General Asset Source API that allows you to add any external library sources you like. This way, you can provide custom assets to your users, such as images, stickers, or anything else.
Check out our Documentation on Custom Asset Source Integration.

Internationalization (i18n)

CE.SDK now supports the easy translation of the default UI into any language, including full i18n support. We ship with English and German locales, but have successfully tested it with other languages.
Learn more about adding translations to CE.SDK in our Documentation.

Headless Mode & APIs

We are finally extending CE.SDK with a full headless mode that is usable without a user interface in a browser. You can use it to load, modify, and export scene templates programmatically without the need for any user interaction. Use it to create previews of your scenes with custom user content without opening the editor, or export your designs in multiple resolutions on demand.

Included APIs

Scene API lets you create, load, save, or export scenes to a rasterized image format.
Block API lets you create, read, update and delete any item (so-called blocks) in the scene; modifying properties like position, size, or content is guaranteed easy.
Layouts API lets you use advanced layout capabilities: choose from modes like absolute positioning, relative positioning in percent relative to the parent, or even auto layout. An even auto-layout lets you auto-adapt the size of a block according to its parent size and siblings.

Other

Snapping now works with multiple selected items.
Slider input values now allow manual input to give you more control.
Easy asset library access in the UI.
Changing content in Preview Mode to preview your template settings does not alter your template anymore.

Upcoming

We are currently working hard to improve the overall CE.SDK Editor experience for you and your customers. That includes adding features like strokes, gradients, grouping, but also improving the rendering performance to create more complex designs. Additionally, we are improving asset discovery within the editor.

Excitingly, we are working hard to bring our VE.SDK video features to CE.SDK! In the near future, you can use videos and animated GIFs inside any template and export short videos from within CE.SDK for Web.

CE.SDK v1.3.0 Release

Daniel — Mon, 17 Jan 2022 15:07:02 GMT

After another six weeks of work, we are happy to ship another update of our CreativeEditor SDK (CE.SDK). This release is again packed with great features to enhance the developer experience and user experience. It includes:

New improved Default UI
Lockable Design & Placeholders
SVG Assets
Multi Selection
Nudging (moving elements with arrow and shift key)
Create scenes from an Image.

New Improved Default UI

That is the first release to introduce a new UI type for CE.SDK, which we call Default UI. It comes with a slick interface and focuses on fast design adoption. The Default UI removes the need for inspectors for many editing cases, thus leaving your users undistracted and focused on adapting or creating designs.

Check our official documentation to learn about configuring the UI Elements.

Lockable Design & Placeholders

In the previous version, CE.SDK introduced the roles “creator” and “adopter” – differentiating between template creation and adoption. In creator mode, you have the liberty to create a design from scratch or change existing designs.

When creators are ready to share a design, they define which design elements can be changed when opening the editor in the “adopter” role. Typically, creators will allow adopters to exchange content, but also the options to style or arrange an item can be controlled.

When the editor is opened in the “adopter” role (Default UI), items that are allowed to change will be highlighted and images will be shown with an overlay indicating that the content shall be replaced before exporting or saving the design.

Find more information about placeholders and constraints in our official documentation.

Dashboard

Creating and adopting designs is one part, but organizing designs is another. With this release, we also introduced a dashboard to create and organize your designs for your users. Reach out to us if you want to see a demo of the dashboard in action and understand.

SVG Assets

With this version, we introduced SVG as an asset-type alternative to JPEG and PNG image formats. This has the benefit that standard SVG Stickers can just be used or uploaded in every design. This has the major advantage that stickers will be crisp at any resolution while coming with a very low download size. Reducing the server load and data transfer for you and your users. In this turn, we also replaced all our example stickers with their SVG equivalents.

Multi Selection

Positioning multiple items at once has been a pain before this release, but not anymore. With the new multi-selection interaction, users can select two or more items at once. Just hold down the “shift” modifier key and start selecting multiple items. Once selected, they can be moved around simultaneously.

Nudging

Always wanted to fine-tune your design and pixel-perfectly align your content?

You can now use the arrow keys (⬅️, ➡️, ⬆️ ⬇️) to nudge the selected items to their correct position pixel by pixel. You can also hold down shift to nudge even further.

Create Scene from Image

Before this release, the CE.SDK could either start from an empty or a predefined scene. From now on, there is also an option to start the editor with any image. It will auto-create a page with the size of the given image and insert the image into this one, allowing a more content-driven creation process. Learn how to create a scene from an image in our documentation.

Minor changes

Besides the major features, there are a ton of small improvements. These include:

UI interface to rename items and pages to a user-defined name
Ability to choose between two UI types, “default” and “advanced”
Searching in image libraries now uses i18n translations
Ability to change the UI Font.

Feel free to send us any questions, or let us know what you think on Twitter.

Thank you for your time!

To stay in the loop with our latest articles and case studies, subscribe to our Newsletter.

Work Where You Work Best

Daniel — Tue, 23 Jun 2020 13:01:57 GMT

When the coronavirus pandemic started spreading in Germany, our team started working from home, weeks before it became mandatory. This step seemed comparatively easy for us, like for many other tech-driven companies.

From the get-go, img.ly has been a company of digital natives, and as such we are used to digital tools for day to day work. Everyone already worked with laptops instead of desktop computers, we used online collaboration tools in the majority of our time, communicated with tools like Slack, and relied on cloud services whenever possible. So when the pandemic hit, we were perfectly equipped to work from home.

As for everyone, there were a few hiccups in the beginning. Many of our teammates missed day-to-day social contact. We all like to chit chat once in a while, enjoy a coffee break in our neighborhood or have lunch together in one of the many local restaurants.

From day one, we added more video calls and played online games to compensate for the missing real-life interactions. We also tried to recreate our office atmosphere by introducing voice channels aside from our other communication channels.

We realized that the newly introduced voice channels were used extensively in the first few weeks but became vacant later on. Instead, people relied on slack or zoom to hop on a call whenever necessary or wrote down problems to discuss with the team asynchronously over time.

After two to four weeks everything seemed to be back at what we would call normal operation. Everyone was back on track and we were making progress towards our goals.

While it is obvious to us that the current situation is pretty special, we realized that it might be a good time to reassess how we work and what defines img.ly.

How can we use the current situation to come out stronger when it’s over?
Somehow it bothered us how everyone talked about things getting back to normal. Obviously, we want our social contacts back, go to restaurants, bars and meet with friends, but what about day-to-day work? Was it better before? Or maybe, does it feel better now?

Work when you work best

Before to this new situation, we always had flexible working hours and relied on everyone making their own decision when to work best. While everyone started at different times during the day, we all came to the office eventually. Working from home was possible but only used as an exception when needed.

The reason was simple: we preferred brief, sometimes spontaneous communication between teams and team members in the office during the day, which made us feel quick in our decision making. Over the years, with a growing team and office space, we sat together to discuss what “our” version of a good working environment was, eventually realizing that there is no one-size-fits-all solution, as everyone has their own needs. That’s why we always thought about different areas in the office that are built to amplify either trivial or deep work. The “virtual office” has similar requirements: We introduced Slack statuses like “Focus work” or “Out for a break” to communicate that they will be getting back later to other teammates if they have any questions. Eventually, everyone seemed to have more control over their own time while being productive, and some are being more productive than before.

Work where you work best

As said, some tensions came up. People were missing real-life contacts, and it seemed that communication between teams could be improved. While the first problem remains a challenge, the latter one got our attention quickly. Before jumping to conclusions, we asked ourselves if this was a new problem, only to realize that it had been there before, just mitigated by the daily run-ins in the office. This resulted in people feeling informed but eventually missing some important details. However, these occasional run-ins in the office didn’t happen anymore, and as such, details would now easily get lost.

Best practice would be to write down all information, record a video, or use other media that can be accessed asynchronously at any time by all team members. However, most saw this as a chore from the management, and as such, it was easily neglected or simply forgotten. Even if the information was written down, it just landed somewhere in our GoogleDrive, Asana, Jira, or some other tool. Principally available for everyone but found and looked at only by some.

But now everyone needs this information as nobody has the occasional run-ins anymore, which creates a general urge to get more information. Instead of lengthy transcripts, people started sharing their results in Slack-channels and updating their daily to-dos in detail. Most importantly, it created awareness and partially converted a chore into something meaningful to the benefit of everyone. However, it is still far from perfect and we will continue to work on that.

Making a temporary state a permanent solution

The taste of increased flexibility in your life and productivity at work leads us to consider making our temporary state a permanent solution.

There are obvious strategic advantages for a tech company: The talent pool increases with a radically increased search radius. Also, we need less office space, a huge pain point we had recently.

Still, there is a yin to the yang, and nothing comes on a silver platter. We now have to filter our talent pool for people that bring the right mindset in terms of expertise, structure, and seniority for remote work. Believe it or not, there are not so many left when applying such filters.

We have to heavily invest in bringing our people together on a very regular basis because we are still a people business, and in-person interactions were key to our company’s success. We know about the perks of being an office-first company. According to Andreas Klinger, “Head of Remote” at AngelList, remote work is great for iterations, while in-person meetings foster innovation. We can see that. It’s all solvable with the right process and mindset, but it’s important to be diligent about this.

Thankfully, this topic has already been extensively covered with conferences, articles, and pioneer companies such as GitLab, Basecamp, and PSPDFKit. I can always recommend them as a reference when you start thinking about becoming a remote company. Of course, it’s easier when you’re a remote company from day one – nevertheless, we are convinced that this change is an opportunity for us to grow and thrive. So in the end, we made the best of the pandemic.

An honest job description – img.ly is hiring a Senior Software Engineer

Daniel — Tue, 19 May 2020 12:14:42 GMT

We are looking for a Senior Software Engineer to join us in building and shaping web-based User Interfaces of our existing and novel products. We are setting up the foundation for a new graphic design tool to challenge the status quo that eases the creation of smashing design and collaboration with fellow creators.

The Job to be done

You will be joining our interdisciplinary team of software engineers and designers and work together with us on our vision to build the creative engine of the digital world. You will mainly focus on further developing our Web Application Frontend of our Photo- and Creative-Editing applications, but also be involved in adjacent engineering tasks and problems. Your job will be to work closely with the rendering/backend team as well as the design team to bring reliable and robust, as well as clean and vivid interfaces to life and help sustain and shape the experience of our User Interfaces.

This position is hands-on ReactJS based UI development ranging from developing reusable components for regular user interfaces as well as on canvas controls for the image-, text-, and design-element manipulation and editing. Also, it includes working with people from multiple fields, brainstorming, and prototyping potential solutions to yet unknown but interesting problems.

The job will require some agility from your side and depending on the current priority you will need to switch between the following working modes: In production your focus should always be on stability, robustness, maintainability, performance, and most importantly good architecture. When prototyping we need fast roundtrips, sensible shortcuts to create proof of concepts and constructive discussions across various stakeholders.

You can expect to work with a team of experts in various fields ranging from rendering to framework building in various programming languages such as C++, JavaScript, Objective-C, Swift, Kotlin, and, for sure TypeScript. Your close teammates are tech enthusiasts with experience in not one but many fields who already have 5 years experience in bringing a successful software development kit to the market and sustain it, and are now working on a new even bigger endeavor.

Here is a list of tasks we have been working on in the team lately. Note, we didn’t work on them all at once and so we don’t expect you to.

Redesigning and implementing our PhotoEditorSDK UI with multiple layouts and extension points, theming and i18n support, and much more.
Conceptualizing and building a new cross-platform creative/graphics rendering engine written in C++ and compiled to WebAssembly.
Building the basis for the User Interface of our new products that use the said engine.
Setting up the whole development environment including rollup and the integration with the WebAssembly backend.
Creating UI components and the design system to ease the development of the User Interface.
Implementing the crop mode and various other image editing UI elements and wiring those to the engine.
Prototyping a concept for dynamic inspector and on-canvas controls (our Gizmos).

What you bring to the Table

We are looking for a candidate who has experience in building web applications, preferably with ReactJS but not necessarily. You should like to be challenged regularly. As your work will touch many areas of our products we hand over a lot of responsibilities to you. We don’t expect you to know everything from the get-go, just openness, and willingness to grow with the team. For this job it’s a plus to have a feeling for good usability and visual design, while you have expert designers helping you, it’s always a plus to be able to design a few things yourself.

At img.ly we like to be able to work autonomously on our tasks and so should you. We keep in touch whenever necessary and everyone is available whenever input is necessary.

We are all generalists, life-long learners, and tech enthusiasts who love what they do. We think and plan before we act while getting things done when needed. We believe that done is better than perfect and iterate after we understand the problem. We all feel responsibility and ownership for what we do, we never let someone down, and we support each other.

We want a candidate that recognizes themself in this description. We have built a team from diverse backgrounds and experiences and see the benefit of that. We don’t beat around the bush and also expect you to be direct and fair.

Long term relationships are important to us, and as such we always look out for each other, we don’t believe in constant crunch modes but in a healthy equilibrium between work and life and enough regeneration times. We believe in work-life blending and as such, we work with people we respect and like and also like to spend quality time with.

We generally don’t believe that certificates like CS degrees are what matters most. We care about what you do and how you do it. Building real stuff counts a lot more for us.

We have an office in Bochum, Germany, but we have always worked from where it’s best for us. We are currently migrating to a remote-first company while still keeping our offices in Bochum, Germany. The caveat is that you must be a good communicator, and as such comfortable with effective writing and on-demand video calls with the team. While you can be anywhere you want, we require you to overlap at least 4 hours with a workday in Germany.

Benefits and Pay

As we like to swap ideas on a daily basis and like to get to know our teammates better, we occasionally meet once or twice a year with everyone at a location to hang out for a week. We visit meetups and conferences and have remote workshops to different topics on a per interest basis. Our goto hardware is a MacBook and whatever everyone feels is necessary for themself.

We work from where we work best and so should you, whether it’s our office in Bochum or your favorite coffee shop next to your home. We value transparency and as such all our numbers are open to everyone in the company. So, last but not least, the salary for this role ranges from 50.000 € to 65.000 € depending on your experience.

How to Apply

We want to integrate people into our team that are as dedicated as we are. That doesn’t mean that we need a novel as a cover letter. A polished CV or your LinkedIn profile can be a safe bet. But, we are far more interested in the projects you’ve been working on, some code, or anything that you want to show us. Rise above the noise. Surprise us.

Our process then is plain and simple. You’ll have an initial chat with us over a cup of coffee to introduce ourselves to each other. Next up is a more sophisticated interview with colleagues from our team about all the technical questions that prey on your mind. In the last step, we set our whole team on meeting you. That means, you’ll have a trial project where you have the chance to get to know our team and our work better, and in turn, we want to see if you fit our motley bunch of people. If everything comes together, we are happy to welcome you to our pack!

If you’re interested, drop us a line to jobs@img.ly

Titles for Software Engineers at img.ly

Daniel — Mon, 18 May 2020 06:39:58 GMT

At IMG.LY, we have simple criteria for our internal titles for a certain job. For each job, we define different stages from beginner to lead roles.
We are aware that not everyone will fit each and every requirement for a stage, however, this guide should help in understanding our work. Our mentality is hands-on and we believe that theory without building something only gets you so far. We iterate, refactor, and rebuild whenever necessary without losing focus of our bigger goal: “Building great products for us and our customers”.

Junior Software Engineer

Needs supervision and back and forth review before finishing a feature/task.
Works on tightly scoped and straight forward features.
Has a basic understanding of architectural patterns.
Uses basic features of programming languages but advanced features have still to be mastered.
Has usually less than two years of experience in professional software development.

Software Engineer

Can work alone on a feature but needs occasional review loopback cycles from a senior.
Can apply common architectural patterns in new environments.
Works mostly on clearly defined features.
Feels responsible for software quality and helps maintain high standards.
Has usually between 2 to 4 years of experience in professional software development.

Senior Software Engineer

Can provide helpful and informative feedback to other Team members.
Can work on vaguely defined features and helps to refine those features with fellow designers and engineers.
Can work in complete autonomy as a sole engineer if necessary on a feature from ideation, over definition to full implementation with the help of a designer, and no further input.
Has a deep expertise in at least a minimum of one programming environment.
Has an understanding of architectural patterns and can apply them on a per need basis.
Feels responsible for software, design, and product quality alike and enforces high standards.
Has usually between 4 to 8 years of experience in professional software development.

Lead Software Engineer

Can work completely independently and autonomously with no need of any kind of supervision.
Helps and supports fellow software engineers to get to work.
Can work on a large chunk of features or complete software application without any kind of supervision.
Is capable of executing projects across multiple domains.
Can manage a team of engineers.
Helps set new standards and introduce new technologies at img.ly.
Has deep knowledge and experience in multiple programming environments.
Has usually between 8 to 12 years of experience in professional software development.
Is an evangelist of his and IMG.LY’s work.

Chief Software Engineer

Is able to set the direction of the entire department, software and product-wise.
Has always a good oversight of cutting-edge technologies and knows when to introduce them beneficially.
Can manage multiple teams of engineers.
Can design, concept, build, and maintain novel software while maintaining and enforcing our standards.
Has usually plus 8 years of experience in professional software development.
Is an evangelist of his and img.ly’s work and shares knowledge at conferences, meetups, social media, and so forth.

Building the Creative Engine of the Digital World

Daniel — Wed, 22 Apr 2020 12:42:48 GMT

Since releasing our first Software Development Kit (SDK) for photo editing in 2015, it quickly found its way into the hands of thousands of application developers. The demand allowed us to ramp up our efforts to expand our SDK to cover more platforms, add features and launch VideoEditor SDK.

At the same time, two very important things happened to us.

First of all, we got experienced in building rendering engines, while learning in-depth about the requirements of processing photos for professional and semi-professional use-cases, on every platform. Our team leveled up big time, to a degree where we are confident there aren’t many teams with such a domain-specific expertise.

Secondly, we gained critical insights into the visual creation process across many different industries, ranging from print services, social networks to marketing tools. In these processes, photo editing is only one part of the creative flow. There are topics like layouting, animation, generative design and many more, which go beyond the mere editing of an image, but are still essential to the business case.

Our vision is to fill the obvious gap, providing a holistic solution that allows users to map entire creative flows into a tool. By leveraging our technology our customers will become a lot faster and more competitive in their markets.

Six months ago, we decided to build the foundation for this vision and kicked off the development of UBQ, an engine empowering a new generation of tools for creativity.

It’s time to shed some light on our creative engine.

What is a Creative Engine?

Before we go forward, let’s establish what we mean by a creative engine. In general, an engine is a platform that allows developers to create tools that support creators to perform specific tasks such as editing photos, creating visual appealing posters, or social media posts. From our perspective, a creative engine will ease the development of design tools for various niches and as such it must be easily adaptable for different usage and user requirements.

To ensure that we don’t have to reinvent the wheel for each creative tool, a creative engine provides building blocks especially targeted to creative output. Some examples for these are:

High-quality image adjustment, editing, filtering, and manipulation.
Automated layouting of design elements on a canvas.
High-quality text rendering and layout algorithms.
Support for various industry-standard assets, image, and video formats.

However, these are only some of the basic features – there are far more advanced things that a creative engine can provide.

On the one hand, this may include scripting tools to ease the generation of generative art or automation for tedious design tasks via automated image segmentation.

On the other hand, there are also tools that not only help the individual creator but ease the cooperation and collaboration between multiple creators and editors. The details may vary, but it can be as simple as defining formats to exchange assets and designs between creators or even allow the simultaneous editing of their creation.

Introducing the UBQ Creative Engine

With UBQ we strive to lay the groundwork for modern visual and communication design.

Believing that these days, a good design tool has to be there for you every step of the way – it has to be ubiquitous**.** Let us explain what this means for us.

Usable on any Platform

In this sense, we want it to be easily accessible to everyone, and as such the UBQ stands for its availability across multiple platforms. It is conceptionally web- and mobile-first but is also available for classic operating systems such as macOS, Linux, and Windows, while being easily portable across various platforms.

Unified Rendering, Generative, and Inference Engine

While our previous products focused on processing a single image or video, our scope is greatly extended with UBQ. Besides being able to process multiple images, with effects such as filters, adjustments, and so forth, in UBQ we generalized images to a concept we call Design Blocks. As such, static images are not the only source of data. It can be anything, even code that generates the visual input. In turn allowing that generative patterns and complex generative art is processed directly in the engine.

We put a lot of work in the last years into modern tools for automation to help creators get rid of time-consuming tasks such as image segmentation, color adjustments, and many others. Our foundation for this stems from modern machine learning and neural network advancements. As such UBQ supports executing neural networks and inferring information. Thus basically allowing any neural network to be part of a design generalized in what we call Compute Blocks*.*

Collaboration – Interact in Realtime with other Creators

The cooperation and collaboration between multiple creators are dear to our hearts. Therefore, we believe that creation is not only the effort of an individual but of many.

UBQ will enable multiple creators to work together. Thus, allowing a design team to work together, share thoughts and assets instantly. Consequently, this constributes in accelerating their workflows and enriching their creative process.

All that is reinforced due to the multiplayer approach to design as well as reusable design elements.

Creators shall be enabled to work together locally as well as remotely and interact on the same design at the same time.

Cooperation – Ease Exchange of Designs with your Peers

Besides the collaborative aspect, UBQ is built around the interaction of creators and non-designers – which we refer to as editors.

Editors will be enabled to take existing designs and adapt it to their needs, by either just giving instant feedback to the creators, changing color palettes to fit the needs of their corporate design guidelines or changing images and text passages to make a design appealing to their audience.

Design Blocks – Smart and Reusable Design Element

Another important design decision is the ability to exchange and reuse design elements between several projects. Therefore, UBQs foundation is built around the concept of smart design elements that we call blocks.

Blocks are reusable design elements that encapsulate complex and tedious design tasks.

They currently come in two flavors, Design and Compute Blocks ranging from simple predefined image adjustments settings to automated image segmentation as well as predefined adaptable design elements.

From our point of view, creators should not have to spend their time with the recreation of preexisting designs. They should not have to watch tutorials just to get to the same output as someone else before. In UBQ, these design blocks are the core foundation for building adaptable and interchangeable designs.

Design blocks respond to the available space, adapting their content accordingly and may include complex logical rules or just a combination of assets. UBQ has all batteries included, providing a rich library of predefined blocks. By design, blocks can easily be built from a combination of other blocks or by using a scripting language similar to processing or p5.js to allow even generative and parametric design.

Final Words

As of April 2020, we are working on the UBQ creative engine for almost half a year now incorporating knowledge from 5+ years developing our SDKs.

We believe we can make an impact on how designs and design-tools are created, adapted and distributed. We are interested in your feedback and open to discussions, also we are looking for people who want to get involved as team members, early adopters or just want to play with it. Expect some demos during the end of summer showcasing what the UBQ engine is capable of.

Soon, we will get back to you with more exciting news.

Stay tuned!

We’re hiring!

Oh, and by the way, we’re hiring a Senior Frontend Developer at img.ly to reinforce our team and help shaping the frontend of our new design tool. Check out the job here and drop us a line, if you are interested.

On the Democratization of Design

Daniel — Mon, 24 Jun 2019 00:00:00 GMT

Over the last century we have experienced the democratization of various domains of our lives:

Key drivers have been technological advancements and lower barriers to leveraging those technologies. In turn, technology became more accessible to more people resulting in products, services, and tools that improve our lives every day.

“Computers are a bicycle for the mind” — Steve Jobs

Tools, such as computers, allow us to finish more tasks in a shorter amount of time and with fewer resources. Even more importantly, said tools also rid us from boring and repetitive work, thus enabling us to shift our attention to exciting and more creative tasks, paving the way for us to experiment and create rather than repeat.

Evolve Design with better Tools

Proper visual design attracts, engages, and persuades; it helps convey stories and facilitates communication.

Nowadays, design in all its facets like advertising, social stories, product presentations, or merchandise is omnipresent in our daily lives. As there is no arguing that appealing and beautiful design has an impact on us and our surrounding, consequently there is also a growing demand for good design and easy-to-use tools.

“Use a picture. It’s worth a thousand words!” — Arthur Brisbamne

Although the importance of design is unquestioned, creating appealing designs remains hard for most of us, and there are many reasons for that.

First of all, design tools are still challenging to master, and while they empower professional designers to create stunning assets, they remain a closed book for most people.

Secondly, even if we leave most of the creational part to professionals, the average user is still unable to use existing designs as a basis to create new assets or refine and update small details. Even for the slightest changes, one has to get familiar with the inner workings of sophisticated design tools.

Also, when it comes to collaboration, the process with most tools is far from frictionless, to put it mildly. Still, most designs get shared as Photoshop, Sketch, or PowerPoint files, and there is no tool or platform that grants unhindered access for multiple stakeholders with varying degrees of experience with design solutions.

As today’s design processes regularly involve multiple stakeholders like designers, marketers, copywriters, or product engineers, one has to acknowledge that prevalent design solutions simply lack features and workflows that reflect daily routines.

Make Design Accessible through Technology

As we pointed out, the overall access to good visual design is still very limited, but with the growing need for compelling creatives in virtually every aspect of the digital and non-digital realm this shouldn’t be and it doesn’t have to. We believe in the democratization of design.

Better tools enable the democratization of design!

We believe that:

We need to overhaul design processes and methods to cope with the ever growing design needs and the ever-rising number of stakeholders.
Achieving elegant design must be made simple.
Design can be made accessible through technology.

The question remains: How do we achieve that?

We came to the conclusion that there are four key elements design tools should have that will make design more accessible:

Collaboration,
building blocks of intelligent design components,
automation of tedious design tasks, and
bringing together Editing, Compositing, and Layouting.

Collaboration

We hold as an opinion that design tools have to facilitate the collaboration between professionals and non-professionals, thus allowing an effective and frictionless design creation and adaption process. This includes direct collaboration on a design as well as the exchange of whole design templates, design components, and presets.

While some solutions already offer templates for content creation, we think that design components as a more modular approach are going to be a necessity to simplify the creation of new complex assets for everyone.

Building Blocks of Intelligent Design Components

If Lego™ taught us anything, then it is that creativity can flow when you base your work on a solid foundation.

That said, one has to admit that while the basic elements of design such as text, shapes, or images can be creatively combined in millions of ways to create stunning designs, they also can be used in a trillion ways to create underwhelming output.

Similar to modern UI design, complex building blocks will play a central role in the visual design process, allowing creators to be more efficient by removing repetitive tasks from the workflow and also allowing average users to create and adopt complex designs quickly. Intelligent design components will shift the focus to the composition by guiding the user in carefully chosen constraints.

Automation of Tedious Design Tasks

Speaking of intelligent building blocks, we believe that said blocks must not be static but adaptive to fit into an effective and frictionless design process. That may include the automation of certain design tasks like text layout or even abstracting away the complexity of image editing.

We envision that with the help of technology and specifically artificial intelligence, recurring and taunting tasks like for example, the separation of a picture into fore- and background can become totally interaction- and hassle-free. Even automatically choosing the right adjustments based on the image content will be possible.

Altogether, intelligent design components are a necessity and will be the next step in design automation, fostering a flexible and fail-safe adaption and creation process and thus ensuring that even without professional knowledge designs will be created faster and look better.

Bringing together Editing, Composition, and Layouting

Photo editing and layouting are both central to the design process. That is why, in a holistic approach, combining both will yield exciting new possibilities, improved visual consistency, and a reduced amount of friction while going back and forth between layout and photo editing.

Closing words

To sum it up: To democratize design, we need intelligent design tools and ecosystems that allow for the cooperation between all stakeholders, as well as a community that is open to share and learn from one another by building and sharing reusable and intelligent design components.

Rebuild or Report?

Daniel — Thu, 26 Apr 2018 00:00:00 GMT

Nowadays, every code project depends on various third party and additional libraries. Truth be told, we wouldn’t be able to create apps or webpages that fast if we would have to build everything from scratch. Typically, we add a library via our favored package manager and give it a go. We start using that specific library, read the docs, figure out how it works and use it for our purposes. Best case, that’s it. Unfortunately, in practice, it may turn out differently.

Make no Mistake, Mistakes happen

While we work with a specific library, we may encounter some roadblocks. Sometimes that’s due to missing or bad documentation. Or it might be due to our missing knowledge about some things. Anyway, sometimes, we eventually realize that it is not us, it was them all along. We found a mistake, a so-called bug.

A (software) bug is an error, flaw, failure or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways.

— Wikipedia

If we’re honest, our first reaction may be that we get angry at the library itself or the author, but, after acknowledging that we never wrote flawless or bug-free code, we look for a solution.

We found a Bug! Now what?

Logically, some of us start to follow the most obvious path. We throw that library away and write a replacement ourselves…not.

Ok, honestly, we may think about that, but it in most cases that is neither the right nor the best way to go. Many authors put a lot of effort into creating software libraries and shared it with us. So, most undoubtedly, a lot of knowledge, work, testing and time has already gone into it.

Therefore, there is a high chance that rewriting that library from scratch is the least favorable option.

So, what should we do?

Worst case scenario: We don’t do anything and leave it be. But this way it won’t get fixed, the authors won’t be aware of it, and others will encounter the same bug and spend unnecessary time on it.

Best case scenario: We track down the bug, propose a solution to the authors and hope they will acknowledge and accept it. But sometimes we don’t have the means, options or time to fix the bug. Consequently, the second best way is telling the authors about the bug you found. But wait, before we file a bug we should ask ourselves the following question

Are we running the latest version of the library?

Writing good Bug Reports helps Yourself and Others

We at IMG.LY provide a software library for photo editing. Thus, we are both library authors as well as users.

I would like to say that we don’t ever make mistakes, but truth be told, everyone does.

As you can imagine, providing photo editing capabilities results in a lot of code for rendering, UI, or other business logic. Although all the usual quality checks are in place, mistakes or unthought-of use-cases manifest in bugs, no matter what.

Consequently, we get bug reports. Now, unfortunately, an awful lot of said bug reports only contain a single line that literally says:

“XYZ does not work.”

While, this is far better than not reporting it at all, it doesn’t help much in hunting down the bug.

With bug reports like this, we spend a lot of our time on trying to reproduce the bug or asking for details on how to reproduce it. That time would be better spent on improving the libraries or helping other users. However, with just a little help we’d be able to find bugs faster and be more effective. So, what should a proper bug report look like? From our point of view, bug reports should be comprised out of the following things:

One bug at a time,
the context of the bug,
a minimal runnable example, and
the steps to reproduce the bug.

One bug at a time

In reality, one may encounter multiple bugs at the same time, and might wish to drop them at the library authors all at once. However, it is far better not to combine multiple problems even if they seem to be alike. So, one should always report each bug individually.

Describing the context

Often forgotten when reporting bugs is to describe the context the bug occurs in. For us, a good bug report contains at least the following specifications:

A description,
the environment
the actual and desired output, and
attachments (e.g. Screenshots).

Description

First thing we need to know is what the bug is about. This includes:

A short meaningful title of the bug (e.g. Exporting image after crop and rotate corrupts output), and
a description of the bug which explains the error in an understandable manner.

Environment

Due to the nature of web and app development, software libraries can be used in many different environments. All of which have their own little differences and quirks. Thus, it is very helpful to include information about the environment, including:

the used operating system’s version (e.g. iOS 10.1),
the device it is running on (e.g. iPhone 7 32GB),
the used browser or runtime environment and its version (e.g. Chrome 65.0.3325.181 (64-Bit) or Node.js 8.11.1 ), and not to forget
the name of the library and its version/variant (e.g. pesdk-html5 4.1.3).

Actual Output and Expected Output

Often, an experienced developer can classify the bug on the basis of the resulting error or error output only. Therefore, a useful bug report should provide:

the actual output that is is observed, and
the desired output that should be generated

Attachments

Additionally, attachments are very helpful to pinpoint the bug. Helpful attachments can be but are not limited to:

error output or logs generated by the compiler and build system, console output, and stack trace, and whenever the bug results in something that is visual recognizable, and
screenshots showing the issue.

Reproducibility is key

While all aforementioned greatly assists to narrow down the context in which the bug appears there is one thing that helps even more:

Even if it seems trivial to reproduce, a good bug report should come with a minimal runnable example that produces the bug in question and a description of how to trigger it.

Providing a minimal running Example

So, one should strip away everything from the current code that isn’t necessary and keep it simple. The faster we and other library maintainers can track down an error, the sooner it’s going to be fixed.

Also, probably it was already necessary to narrow down the issue so stripping it out into a minimal example is not too much work. If possible, the Gist should be provided via a GitHub repository or any other similar sharing system.

Keep it simple to trigger the Bug and be specific about it

While having a minimal example is already lovely, sometimes it needs a bit more than just code to trigger the bug. Writing down precise instructions on how to trigger it is not always easy but being specific dramatically helps to get to it.

That is why one should summarize the problem in brevity and with utmost precision.

Conclusions

From our point of view these are the most important things one needs to consider when reporting a bug. Most certainly, the requirements change and need to be adapted for different types of software. We’d really like your feedback on this. So, if you have any comments and additions please let us know.

**Thanks for reading! To stay in the loop, subscribe to our Newsletter.**

Build or Buy?

Daniel — Thu, 02 Nov 2017 00:00:00 GMT

For many years, we at IMG.LY and our partners at 9elements have been developing various applications for customers and ourselves. Often, we had to decide whether to build everything from scratch or to consider third party components. Sounds familiar?

As developers, makers and tinkerers we often tend to build everything ourselves. It seems to be in our nature and part of who we are. Truth be told, this is the point where we often got irrational.

Taking a good look at modern application development, we quickly realize that everything is getting more complex in a tremendous pace. While it was feasible to develop apps without any third party libraries back then, it has become more or less of a standard to employ third party libraries or tools that solve certain problems for us. Nowadays, there are a lot of features that are must haves and simply expected to be in your product.

Thus, we are more aware than ever when it comes to the decision whether to build or buy.

Decisions

Most often, our resources are not endless. With limited time, money, and labour power we always struggle to make the decision whether to build a certain component of our software ourselves or buy it from a third-party.

But why do we struggle? Are we sure we can do it better than everybody else? Are we not trusting the third-party? Do we just think it is cheaper to do it ourselves? Or, do we just underestimate the amount of work that is required to build that one component?

Most certainly we know that software, even a small app, can be a complex beast that is comprised out of a lot of components. Even more surely, our idea or product should stand out of the masses. So, should we really spend too much time building a component or rather focus on getting things done efficiently? How do we set the priorities?

Priorities

Probably, we want to focus on the things that set our product apart from all others, may it be a singular specific feature or a novel combination of already existing ones.

Surely, most of us will never consider writing their own relational database like PostgreSQL. But, what about features such as payment, authentication or a content management system?

Speaking from experience, most of us won’t even think about implementing payment themselves and eventually, we will use a popular SaaS such as Stripe. But, what about authentication? Would we build that ourselves?

Thinking about all the details that you need to account for when building an authentication system, it becomes obvious that more is needed than just providing a simple registration with username and password. Depending on the software we are developing, people might expect to be able to login with their social logins via e.g. Facebook or GitHub. Also, we would need to provide a way to reset passwords, send emails on registration and so on. Over time, the list of requirements will grow and we will be forced to spend our precious resources on it, whether we want it or not.

So, why not just let someone else do the job for you and solve all these problems?

Don’t we trust services like Auth0 to solve all these problems for us, or what is the reason we build those things ourselves? Evidently, if our product is an authentication system we should build it ourselves. But, let’s be honest, most of the time it is just something we need for our product and nothing we care about deeply.

Unfortunately, we know that everything comes at a price. In case of Auth0, the login data may not be any more in our possession. However, are we sure that it is really bad? We tell ourselves that we can protect our data in a way that no one else would have access, no data would be leaked and so on. At least, that is what we would be thinking, right? But, are we sure that we can protect the data better than the service whose one and only task is to care about exactly that? And what about backups? Do we really want to put our resources into creating and checking that our data is properly backed up and restorable in case of a server failure? Probably we won’t and in many cases it would be a good idea to let someone do the job who does it on a day to day basis. Being completely honest with ourselves, we realized that:

By zeroing in on one specific problem, we can build better and more advanced tools than we could by trying to solve multiple problems at once.

We experienced first hand how much time and effort is required to tackle one specific problem when we needed a photo editing solution as part of one of our projects. In that particular case, we needed to go down the rabbit hole and build it ourselves. Back then, we didn’t have any other option as there was no component available that we could have used.

Was it a good idea? Probably not for the project but while building the components we realized that others could benefit from our learnings. This is basically why we started building and advancing our PhotoEditor SDK.

Apart from the previous mentioned pros and cons about building and buying, there is also another important factor: These services cost money and most of us will look at the pricing and believe it may be too expensive. Especially, if you have to take out your credit card and start a subscription that may feel like slowly draining your bank account. Even more so if you’re a small business or startup and every additional expense poses a threat to your project as a whole. However, this is the time to really weigh your options.

Time is Money

We always hear and rephrase the good old time is money slogan. And this is true in many ways. Even if we don’t care much about the cost of our own time, we would still have to invest it to build said feature and by that might delay the whole product. Do we really want to spend that time and delay our whole product?

Ok, assuming we don’t do it ourselves and we have a team of developers and parallelize our efforts, we still need to pay for it with, and let’s be honest … money.

According to PayScale the average salary of a software developer is around 45k€ in Germany. Considering that we work around 220 days per year, each day would at least cost us approximately 200€, but to be fair it will surely be more. Depending on the software component you are considering to build or the SaaS we consider to use, we most surely would get a lot more for our 200€ than what we could build in one day. And, if we are honest, 200€ are definitely not enough to develop any feature that requires more than just a few lines of code.

Final words

Summing it up, we should definitely not just buy every component from a third party or subscribe to every SaaS, but we should weigh carefully in which cases we really have to build it ourselves or when buying is the better option.

The whole point is that whether you buy some of your components or build everything yourself really doesn’t matter as long as it’s the smartest decision for your business. And if you automatically shy away from solutions that cost you something and decide to build it yourself no matter what, that decision might cost you a whole lot more.

The good thing is, that there is a trend that third parties tend to focus on more granular problems, which allows us to make the distinct decision whether to build or buy a specific component of our application.

Thanks for reading! To stay in the loop, subscribe to our Newsletter.

Daniel – IMG.LY Blog

One SDK to Power Them All: Announcing IMG.LY SDK

From Fragmentation to Unification

One SDK to Power Them All

Long-Term Support for PE. and VE.SDK

What’s Next?

Vibe-Engineering: When AI Does All the Coding, What Do We Actually Do?

The Experiment: AI Agents Code, Humans Curate

The General Workflow That Emerged

Quality Assurance & Knowledge Capturing Matters More Than Ever

Tips & Tricks

AI Is Eager, Resourceful, and Has Memory Like a Sieve

Further Remarks

Conclusion

Afterthought

Try It Yourself

Appendix

Appendix: Claude Code Wishlist

Appendix: Most Used Prompts

Appendix: Claude Code Settings

Library Capabilities

Background Removal in the Browser Using ONNX Runtime with WebGPU

Towards Real-time Background Removal in the Browser

Implementation Details

Evaluation

First Use – Neural Network Download

First Run – Neural Network Initialization / Compilation

Consecutive Runs – Neural Network Execution

Conclusion

Unleash Creativity with CE.SDK’s New Plugin System

Built for Modification

Unlocking the AI Revolution

Boosting Customer Autonomy

Accelerate Product Expansion

Key Concepts

What Can You Build with our Plugin System?

Custom Actions

Custom Tools

Custom Assets & Presets

Custom Libraries

Custom Editor Behavior

Custom User Feedback

What’s Next

CE.SDK v1.7.0 Release and Roadmap

CE.SDK v1.7.0

Auto Close Option in Web UI Configuration for Asset Libraries

Custom URI Resolver

Improving Mobile Touch Support

Rotation of Groups and Hierarchies

Custom Asset Source UI

Moving from Roles to Abilities: Disconnecting UI Variants from Roles

Shadows in Advanced UI

Blur API

Effects API

Image Fills API

Image Straighten API

Improved Group Selection and API

Scopes API

Shadows API

Zoom API

Product Roadmap

Thanks for reading! Let us know what you think on Twitter! To stay in the loop, subscribe to our Newsletter.

CE.SDK v1.6.0 Release

Linear Gradient Fills

Strokes, Outline, Insets, and Stroke Patterns

Blend Modes

Scale Groups, Multi-Selection, and Hierarchies

UI Template Library

Floating and Static Asset Library Panels

Placeholders API

Editing State APIs

API for Image Fit Modes

Thanks for reading! Let us know what you think on Twitter, or stay in the loop with our Newsletter.

CE.SDK v1.5.0 Release

Hierarchies (Groups)

Template Imports & Exports

More Performance

Improved Touch & Mobile Support

PDF Export

Build Your Custom UI