Skip to main content
Software Engineering

Return to Question

Tweeted twitter.com/StackSoftEng/status/1164326436771155969
edited body
Source Link
lovasoa
lovasoa
  • 199
  • 4

Tiled images

Tiled images are large images that have been split in smaller square tiles. There are several tiled image formats, with different ways of organizing the tile files. A tiled image on the web can be downloaded only by finding all the individual tile URLs, downloading them, and stitching them back together. This is what a tiled image downloader does.

zoomable image

Building a tiled image downloader

I maintain several tiled image downloaders (mainly dezoomify, and dezoomify-rs). Their implementation is quite simple, the only software architecture challenges are to:

  1. Make as much of the software as possible testable without having to actually make network requests
  2. Support many different tiled image formats without duplicating code.

I call dezoomers the modules of the software that are specific to each tiled image format. The goal is to be able to have dezoomers be as small as possible, and to have the core of the software do most of the work.

Implementing template-only tiled image download

Currently, in dezoomify-rs, the dezoomers provide a list of tile URLs and associated positions, and then the core downloads and stitches the tiles in parallel. That works for most image formats, but not for template-only downloads.

Template-only download is a feature that would allow the software to take as a single input an URL template of the form http://test.com/{x},{y}.jpg, and be able to replace {x} and {y} by coordinates to create the image. The challenge here is that the bounds for x and y are not known in advance, and can be computed only by requesting tiles, and looking at the response of the server: as soon as the server returns a 404, we know that we have reached the maximum value for x, and process to the next line, until the server returns a 404 even for x=0, at which point we know we have reached the maximal value for y.

Here is a simple schema I made of the state machine that could be implemented for template-only downloads:

state machinestate machine

The question is: how to architecture the software in a way that:

  1. allows both template-only downloads and traditional dezoomers when all the tile URLs are known in advance.
  2. is efficient (always downloads as many tiles in parallel as possible)
  3. is testable
  4. avoids code duplication

Tiled images

Tiled images are large images that have been split in smaller square tiles. There are several tiled image formats, with different ways of organizing the tile files. A tiled image on the web can be downloaded only by finding all the individual tile URLs, downloading them, and stitching them back together. This is what a tiled image downloader does.

zoomable image

Building a tiled image downloader

I maintain several tiled image downloaders (mainly dezoomify, and dezoomify-rs). Their implementation is quite simple, the only software architecture challenges are to:

  1. Make as much of the software as possible testable without having to actually make network requests
  2. Support many different tiled image formats without duplicating code.

I call dezoomers the modules of the software that are specific to each tiled image format. The goal is to be able to have dezoomers be as small as possible, and to have the core of the software do most of the work.

Implementing template-only tiled image download

Currently, in dezoomify-rs, the dezoomers provide a list of tile URLs and associated positions, and then the core downloads and stitches the tiles in parallel. That works for most image formats, but not for template-only downloads.

Template-only download is a feature that would allow the software to take as a single input an URL template of the form http://test.com/{x},{y}.jpg, and be able to replace {x} and {y} by coordinates to create the image. The challenge here is that the bounds for x and y are not known in advance, and can be computed only by requesting tiles, and looking at the response of the server: as soon as the server returns a 404, we know that we have reached the maximum value for x, and process to the next line, until the server returns a 404 even for x=0, at which point we know we have reached the maximal value for y.

Here is a simple schema I made of the state machine that could be implemented for template-only downloads:

state machine

The question is: how to architecture the software in a way that:

  1. allows both template-only downloads and traditional dezoomers when all the tile URLs are known in advance.
  2. is efficient (always downloads as many tiles in parallel as possible)
  3. is testable
  4. avoids code duplication

Tiled images

Tiled images are large images that have been split in smaller square tiles. There are several tiled image formats, with different ways of organizing the tile files. A tiled image on the web can be downloaded only by finding all the individual tile URLs, downloading them, and stitching them back together. This is what a tiled image downloader does.

zoomable image

Building a tiled image downloader

I maintain several tiled image downloaders (mainly dezoomify, and dezoomify-rs). Their implementation is quite simple, the only software architecture challenges are to:

  1. Make as much of the software as possible testable without having to actually make network requests
  2. Support many different tiled image formats without duplicating code.

I call dezoomers the modules of the software that are specific to each tiled image format. The goal is to be able to have dezoomers be as small as possible, and to have the core of the software do most of the work.

Implementing template-only tiled image download

Currently, in dezoomify-rs, the dezoomers provide a list of tile URLs and associated positions, and then the core downloads and stitches the tiles in parallel. That works for most image formats, but not for template-only downloads.

Template-only download is a feature that would allow the software to take as a single input an URL template of the form http://test.com/{x},{y}.jpg, and be able to replace {x} and {y} by coordinates to create the image. The challenge here is that the bounds for x and y are not known in advance, and can be computed only by requesting tiles, and looking at the response of the server: as soon as the server returns a 404, we know that we have reached the maximum value for x, and process to the next line, until the server returns a 404 even for x=0, at which point we know we have reached the maximal value for y.

Here is a simple schema I made of the state machine that could be implemented for template-only downloads:

state machine

The question is: how to architecture the software in a way that:

  1. allows both template-only downloads and traditional dezoomers when all the tile URLs are known in advance.
  2. is efficient (always downloads as many tiles in parallel as possible)
  3. is testable
  4. avoids code duplication
Source Link
lovasoa
lovasoa
  • 199
  • 4

How to design a good generic tiled image downloader?

Tiled images

Tiled images are large images that have been split in smaller square tiles. There are several tiled image formats, with different ways of organizing the tile files. A tiled image on the web can be downloaded only by finding all the individual tile URLs, downloading them, and stitching them back together. This is what a tiled image downloader does.

zoomable image

Building a tiled image downloader

I maintain several tiled image downloaders (mainly dezoomify, and dezoomify-rs). Their implementation is quite simple, the only software architecture challenges are to:

  1. Make as much of the software as possible testable without having to actually make network requests
  2. Support many different tiled image formats without duplicating code.

I call dezoomers the modules of the software that are specific to each tiled image format. The goal is to be able to have dezoomers be as small as possible, and to have the core of the software do most of the work.

Implementing template-only tiled image download

Currently, in dezoomify-rs, the dezoomers provide a list of tile URLs and associated positions, and then the core downloads and stitches the tiles in parallel. That works for most image formats, but not for template-only downloads.

Template-only download is a feature that would allow the software to take as a single input an URL template of the form http://test.com/{x},{y}.jpg, and be able to replace {x} and {y} by coordinates to create the image. The challenge here is that the bounds for x and y are not known in advance, and can be computed only by requesting tiles, and looking at the response of the server: as soon as the server returns a 404, we know that we have reached the maximum value for x, and process to the next line, until the server returns a 404 even for x=0, at which point we know we have reached the maximal value for y.

Here is a simple schema I made of the state machine that could be implemented for template-only downloads:

state machine

The question is: how to architecture the software in a way that:

  1. allows both template-only downloads and traditional dezoomers when all the tile URLs are known in advance.
  2. is efficient (always downloads as many tiles in parallel as possible)
  3. is testable
  4. avoids code duplication