Photos alias when shrunk and turn blocky under camera zoom: image and video sample with nearest neighbour and no mipmaps

[ThomasTartrau]

image and video draw their bitmap with canvas.draw_image_rect(img, None, dst, &Paint::default()), which samples with Skia's default SamplingOptions: nearest neighbour, no mipmaps. A photo drawn smaller than its source picks one source pixel out of N and drops the rest, so it aliases (jagged hair, stair-stepped glasses, moiré on fabric). A photo drawn larger, for instance under a camera zoom, is enlarged pixel by pixel into visible blocks.

crates/rustmotion-components/src/image.rs:

let dst = Rect::from_xywh(offset_x, offset_y, draw_w, draw_h);
let paint = Paint::default();
// ...
canvas.draw_image_rect(img, None, dst, &paint);

video.rs (draw_fitted) has the same call. The camera is applied as a canvas matrix (apply_camera_transform scales the canvas), so the scale the image is finally drawn at is the layout scale times the camera zoom, and both go through that nearest-neighbour sampling.

Reproduction

A 640x640 checkerboard of 1px black/white cells (checker.png), drawn at 64px and 120px on a #808080 background. Correct downsampling averages it to the background grey.

{
  "version": "1.0",
  "video": { "width": 320, "height": 160, "fps": 30, "background": "#808080" },
  "scenes": [{
    "duration": 1.0,
    "layout": { "direction": "row", "align_items": "center", "justify_content": "center", "gap": 40 },
    "children": [
      { "type": "image", "src": "checker.png", "fit": "cover", "style": { "width": 64, "height": 64 } },
      { "type": "image", "src": "checker.png", "fit": "cover", "style": { "width": 120, "height": 120 } }
    ]
  }]
}

Luma inside each image (4px inset), render --frame 0:

drawn size	mean	min	max	near-black or near-white pixels
64px (current)	255	255	255	100%
120px (current)	124	0	255	100%
64px (mipmapped)	127	127	127	0%
120px (mipmapped)	127	127	127	0%

At 64px the scale is exactly 1/10, so every sampled pixel lands on the same parity and the image renders solid white. At 120px it is black/white noise.

Seen in production on a video using portraits: 640px faces drawn as 52px avatars in a table had visibly jagged hair and glasses, and a wall of 400px portraits under a zoom: 3.4 camera move showed pixel blocks.

Proposed fix

Choose the sampling from the on-screen size, camera included:

• drawn smaller than the source: SamplingOptions::new(FilterMode::Linear, MipmapMode::Linear);
• drawn at or above source size: SamplingOptions::from(CubicResampler::mitchell()) (cubic does not use mipmaps, so it is only right when enlarging).

The on-screen width is dst.width() times the scale of canvas.local_to_device_as_3x3(), which already includes the camera zoom. A shared helper in rustmotion-core::engine::renderer (draw_photo(canvas, image, dst, paint)) can replace the draw_image_rect call in image.rs and video.rs. A local patch along those lines gives the "mipmapped" rows above. icon.rs already does something similar with CubicResampler::mitchell().

The other places that draw a bitmap with Paint::default() are not covered by that patch and likely have the same problem: avatar.rs, avatar_group.rs, badge.rs, list.rs, stat.rs, mockup.rs, gif.rs, svg.rs (raster fallbacks), lottie.rs, engine/transition.rs and the sheet command.

[LeadcodeDev]

Working on this.

[LeadcodeDev]

Mini-spec

Goal. One sampling decision, taken from the on-screen size the bitmap is finally drawn at, applied everywhere a raster bitmap is scaled to a destination rect.

Shape. renderer::photo: photo_sampling(source, on_screen) -> SamplingOptions (pure, mipmapped linear when either axis minifies, Mitchell cubic otherwise) plus draw_photo(canvas, image, dst, paint), which reads the on-screen size off canvas.local_to_device_as_3x3() so the camera zoom is included. The axis scales come from hypot(scale_x, skew_y) rather than scale_x alone, so a rotated matrix does not report a shrink that is not there.

Invariants.

• A bitmap drawn at or above source size never samples with mipmaps (cubic does not use them).
• The sampling never depends on anything but source size and on-screen size, so the same node under a camera and without one is decided on the same rule.
• No call site decides for itself.

Scope. 14 call sites: image, video, avatar, avatar_group, badge, list, stat, mockup, gif, lottie, svg raster fallback, icon, transition::render_layer, and the sheet command.

Out of scope, with the reason:

• transition.rs:273 and engine/render/scene.rs:207 composite a full frame buffer at 1:1 through the point form of draw_image. Nothing is scaled, so the sampling is never consulted.
• paint_pass.rs:934 draws a layer snapshot, also through the point form. It is under a scaling matrix, so it has the same defect, but a snapshot of already-rasterised content is a different question from a photo and the point form takes no dst rect. Left out deliberately.
• engine/text/cosmic.rs:196 is the cosmic-text bridge, which CLAUDE.md records as not wired into real rendering.

Acceptance.

1. A 640px checkerboard drawn at 64px (scale 1/10) reads mean luma 127 ± 2 with no near-black or near-white pixel, against 255 solid white measured on main.
2. The same at 120px reads mean 127 ± 2, against mean 124 with min 0 / max 255 on main.
3. A 640px source drawn into a 640px rect on a canvas scaled to 0.1 picks the minifying path: the camera zoom is read, not just the layout size.
4. An enlargement picks cubic, not mipmaps.

Verify (exit). cargo fmt --all --check; cargo clippy --workspace --all-targets --features rustmotion/studio -- -D warnings; cargo test --workspace --features rustmotion/studio.

Risk. Mipmap generation costs memory and time per image on first draw. Measured before the PR goes up.

[LeadcodeDev]

Fixed in #468.

Twenty call sites rather than the eleven listed: video, lottie and svg each had more than one, and transition::render_layer is reached through surface.canvas() so a grep for canvas.draw_image_rect misses it. icon went through the helper too — it already asked for Mitchell, but unconditionally, so a shrunk icon had no mipmaps.

Verified: the checkerboard reads mean 127 at both 64px and 120px where it read 255 solid and 0-to-255 noise before, measured through the image component end to end and confirmed red before the fix. Two of the unit tests assert the old path's numbers so the fix cannot quietly become a no-op. A 640px rect on a canvas scaled to 0.1 confirms the camera zoom is read, not just the layout size. Full suite: 2153 passed, 0 failed.

Measured cost, 300 frames at 1920x1080: twelve photos at 52px go 1.01s to 1.13s, one 640px source filling the frame goes 1.79s to 2.59s. The enlargement is where Mitchell is paid for.

Not done, and named in the PR: paint_pass.rs:934 draws a layer snapshot under a scaling matrix through the point form of draw_image, so it has the same defect. A snapshot of already-rasterised content is a different question from a photo and the point form takes no dst rect, so it is left out rather than guessed at. Say if you want an issue for it.