# Python : Media : ImageIO \[ [src](https://github.com/imageio/imageio/) | [docs](https://imageio.readthedocs.io/en/stable/) | [API](https://imageio.readthedocs.io/en/stable/reference/) ] Focused on IO and conversion, not so much creation and transformation. Supports animated images, video, volumetric data, and scientific formats. (295 total) **Dependencies** - images: NumPy + Pillow - videos: [[Tech/Python/Media/Overview|PyAV]] or [[imageio#imageio-ffmpeg|imageio-ffmpeg]] ## Concepts The core of ImageIO is a set of user-facing APIs combined with a plugin manager. API calls choose sensible defaults and then call the plugin manager, which deduces the correct plugin / backend to use for the given resource and file format. The plugin manager adds sensible backend-specific defaults and then calls one of ImageIOs many backends to perform the actual loading. This allows ImageIO to take care of most of the gory details of loading images for you, while still allowing you to customize the behavior when and where you need to. You can think of ImageIO in three parts that work in sequence in order to load an image: 1. **ImageIO Core** — v3 API + plugin manager; intelligently chooses a matching plugin if not specified explicitly 2. **Plugin** — backend-facing adapter / wrapper; converts request from core into sequence of backend instructions (read/write/iter/etc) 3. **Backend** — can read and/or write images or image-like objects (like videos) **Backends:** Pillow, ffmpeg, etc **ImageResource** -> A blob of data that contains image data. ## Cheatsheet #### Introduction ```python import imageio.v3 as iio im = iio.imread( "imageio:chelsea.png" ) # read a standard image im.shape # im is a NumPy array of shape ( 300, 451, 3 ) iio.imwrite( "chelsea.jpg", im ) # convert to jpg ``` > [!NOTE] Example Images > Filenames beginning with "imageio:" refer to built-in files which are useful for testing. #### API ```python imread( uri, index, plugin, extension, **kwargs ) # read imwrite( uri, image, plugin, extension, **kwargs ) # write imiter( uri, plugin, extension, **kwargs ) # iterate image sequence (videos & animations) improps( uri, index, plugin, extension, **kwargs ) # standardized metadata immeta( uri, index, plugin, extension, **kwargs ) # format-specific metadata imopen( uri, io_mode, plugin, extension, **kwargs ) # open an ImageResource for advanced usage extension : treat provided ImageResource as if it had the given extension (ex: "png") index (r-only) : index of the ndimage to read plugin : name (ex: "pyav") uri : "filename" | "url" | fileobj | "<screen>" | "<clipboard>" "<screen>" # docs says works on Win & OS X, doesn't mention linux # when tried: PIL/ImageGrab.py: OSError: "Pillow was built without XCB support" # (using v11.1.0 via conda) "<capture>" # works if image in clipboard, else: # RuntimeError: "There seems to be no image data on the clipboard now." Additional kwargs are specific to the plugin being used. # Frame API (np.ndarray) frame.shape Tuple[int, …] # The shape of the loaded ndimage. frame.dtype np.dtype # The dtype of the loaded ndimage. frame.n_images int # Number of images in the file if index=..., None for single images. frame.is_batch bool # If True, the first dimension of the ndimage represents a batch dimension along which several images are stacked. frame.spacing Tuple # A tuple describing the spacing between pixels along each axis of the ndimage. # If the spacing is uniform along an axis the value corresponding to that axis is a single float. # If the spacing is non-uniform, the value corresponding to that axis is a tuple in which # the i-th element indicates the spacing between the i-th and (i+1)-th pixel along that axis. ``` #### Cookbook ```python # read a single frame frame = iio.imread( "imageio:cockatoo.mp4", index = 42, plugin = "pyav" ) # bulk read all frames (will consume a lot of memory) frames = iio.imread( "imageio:cockatoo.mp4", plugin = "pyav" ) # iterate over large videos for frame in iio.imiter( "imageio:cockatoo.mp4", plugin = "pyav" ): print( frame.shape, frame.dtype ) # re-encode video with iio.imopen( "output.mkv", "w", plugin = "pyav" ) as out_file: out_file.init_video_stream( "vp9", fps = fps ) for frame in iio.imiter( "input.mp4", plugin = "pyav" ): out_file.write_frame( frame ) # grabbing frames from webcam (works for me on ubuntu/conda - was 640x480 when saved as png) for idx, frame in enumerate( iio.imiter( "<video0>" ) ): print( f"Frame { idx }: avg. color { np.sum( frame, axis = -1 ) }" ) ``` #### Metadata Format-specific metadata comes in the form of a python dict and aims to expose all the metadata contained in the image using the containers / plugins key and format: ```python metadata = iio.immeta( "imageio:chelsea.png" ) print( metadata[ "mode" ] ) # "RGB" fps = iio.immeta( source, plugin = "pyav" )[ "fps" ] ``` Standardized metadata, on the other hand, comes in the form of the `imageio.core.v3_plugin_api.ImageProperties` dataclass and aims to expose a curated set of metadata using a standardized name and format independent of the underlying container or plugin: ```python props = iio.improps( "imageio:chelsea.png" ) print( props.shape ) # ( 300, 451, 3 ) print( props.dtype ) # dtype( "uint8" ) ``` #### Example Images ``` astronaut.png Image of the astronaut Eileen Collins. camera.png A grayscale image of a photographer. checkerboard.png Black and white image of a chekerboard. wood.jpg A (repeatable) texture of wooden planks. bricks.jpg A (repeatable) texture of stone bricks. clock.png Photo of a clock with motion blur (Stefan van der Walt). coffee.png Image of a cup of coffee (Rachel Michetti). chelsea.png Image of Stefan's cat. wikkie.png Image of Almar's cat. coins.png Image showing greek coins from Pompeii. horse.png Image showing the silhouette of a horse (Andreas Preuss). hubble_deep_field.png Photograph taken by Hubble telescope (NASA). immunohistochemistry.png Immunohistochemical (IHC) staining. moon.png Image showing a portion of the surface of the moon. page.png A scanned page of text. text.png A photograph of handdrawn text. bacterial_colony.tif Multi-page TIFF image of a bacterial colony. chelsea.zip The chelsea.png in a zipfile (for testing). chelsea.bsdf The chelsea.png in a BSDF file(for testing). newtonscradle.gif Animated GIF of a newton's cradle. cockatoo.mp4 Video file of a cockatoo. cockatoo_yuv420.mp4 Video file of a cockatoo with yuv420 pixel format. stent.npz Volumetric image showing a stented abdominal aorta. meadow_cube.jpg A cubemap image of a meadow, e.g. to render a skybox. ``` ## imageio-ffmpeg Created before PyAV. Author recommends PyAV instead — faster & more features. #### Overview - comes with 80mb statically-compiled ffmpeg binary - provides two simple generator functions for reading from / writing to ffmpeg which reliably terminate the proces when done - calls ffmpeg in a subprocess, and video frames are communicated over pipes - lots of options for both functions #### Cheatsheet ```python from imageio_ffmpeg import count_frames_and_secs, get_fmpeg_exe, get_ffmpeg_version, read_frames, write_frames for frame in read_frames( path ): ... # frame : bytes writer = write_frames( path, size ) # size = ( w, h ) writer.send( None ) # seed the generator ??? for frame in frames: writer.send( frame ) writer.close() count_frames_and_secs( path ) -> ( 300, 10.1 ) get_ffmpeg_exe() -> "/home/user/anaconda3/envs/fs3/lib/python3.11/site-packages/imageio_ffmpeg/binaries/ffmpeg-linux-x86_64-v7.0.2" get_ffmpeg_version -> "7.0.2-static" ```