Responsive Images: The Incomplete <picture>

If there's one thing that there've been an ample number of articles telling front end developers they must unequivocally adopt in the last five years, it's responsive images. We're all aware that more and more users are viewing our websites on mobile devices, and standard definition image assets simply won't do for today's retina displays. Thankfully, the W3C and WHATWG standards bodies have done a great job of taking feedback to give front-end developers simple but powerful tools to begin handling serving images at a wide range of resolutions for all the devices we now have to target. However, while eager web developers have been pushing everyone to add these new tools to their toolkit, recent projects have taught me that there are quite a few caveats that the considerate developer should be aware of before they start serving responsive images.

Firstly, let's discuss what tools currently exist and when to use them. The two main responsive image solutions added in HTML5 are img srcset and the <picture> tag. <picture> is a container for a set of image <source>s, where each source provides srcset attribute with a path to the image you want to show and a media attribute with a standard media query breakpoint. You can list as many <source>s as you wish to have images, and each can also choose between a 1x or 2x size image for retina screens. Finally, there's a normal <img> tag at the end of your <source>s with all the standard src and alt attributes that you'd normally include. Regardless of what resolution you're viewing the page at, the <img> tag is actually the only rendered element, and all the <source>s do is swap out the src attribute in the <img> tag. Also, in the case that the browser doesn't support responsive images, the <img> tag is a safe fallback. If you're going to do any extra styling on your picture, just apply it to the <img> tag. An example of <picture> looks like this:

Img srcset looks much more compact, as it only needs a single <img> tag to contain all your sources. It'll have a standard src and alt attribute, but it'll add the srcset and sizes attributes as well. srcset contains a comma separated list of your different resolution images and the actual image width. You must supply the width here because the browser won't know what the width of each image is until it first retrieves it, so to avoid downloading each one, we give the browser a hint. sizes& is the viewport width of the image, that is, how much of the browser width the image is going to take up on the page. An image that spans the entire width of the page is 100vw, while an image that floats to the side of your content might be only 15vw. Nowhere here have we told the browser at what breakpoints it should retrieve each image source, because we're leaving it up to the browser to decide, based on how much space we expect each image to take up, the network speed of the device, and whether it may have already cached a higher resolution version of the same image. However, we could use a media query to tell the size attribute that an image that's 50vw on a desktop is 100vw on a phone. Again, the plain src attribute gets swapped out behind the scenes and serves as a fallback for older browsers. Img srcset looks like this:

So, with two very different approaches to responsive images, which one should you use and when? The answer to this question took me quite a long time to figure out, despite there being a lot of articles devoted to this topic. Most articles will say that <picture> should be used for "art direction", but as a non-designer, that meant nothing to me. So, in layman's terms, what this really means is, if changing the source of your responsive image wouldn't change the way the page is displayed at all, except for the resolution of the image itself, it's safe to use img srcset. img srcset is smarter because, all things being equal, it lets the browser decide. However, if you're requiring a specific image at a specific breakpoint, like for example, because the images are cropped differently, or are a different aspect ratio, or otherwise look a specific way to display properly at a fixed resolution, then <picture> element will give you the best results.

Incidentally, you can mix-and-match both <picture> and img srcset by having a srcsetin the ;<img> tag at the end of your <picture>. This may be useful if, for example, art direction is only important above or below certain breakpoints. Also, there is an equivalent to img srcset, but for the CSS background-image property called image-set() which is doesn't have a final draft specification yet, but which has already been adopted by some browsers.

Now, with a working understanding of the great options out there for responsive images, there's nothing holding us back from making fast, scalable, responsive websites, right? If you've been reading the same articles on responsive web design best practices that I have for the last few years, you'd be led to believe that's true, but not so fast! Would you be surprised to learn that Safari, the browser of choice for most companies targeting a mobile redesign, doesn't support the very useful <picture> element? And as of the time of writing this, nor do the unreleased versions of Safari in iOS 9, or Microsoft's Edge browser. Firefox only just added it in the most recent version, and Chrome has supported it for a while on both desktop and mobile devices. img srcset is more promising, but is still only partially implemented in Safari and Edge. And while image-set() has long been supported in all versions of Chrome and Safari, there are no plans to incorporate it into Edge or Firefox at this time, nor should there be while the spec is still undergoing drastic changes. Oh, what a travesty! How will we get cross-platform responsive images on all our devices?

Paraphrasing what should become the new front-end motto, there's a polyfill for that. Introducing Picturefill, a Javascript plugin that brings both <picture> and img srcset support to any browser that understands basic HTML5 and CSS3, such as IE9 (or even earlier versions of IE with an additional polyfill). Picturefill uses the current standard syntax as of version 2, so you can very nearly write your responsive image tags assuming any browser supports it natively and you won't need to make any changes as browsers that currently depend on the polyfill add native support.

However, as noted on the Picturefill website, there are a few browser-specific caveats. (You didn't think we'd get this far and have it be easy, did you?) For example, while IE9 has a level of HTML5 understanding, it only understands the <source> tag in the context of providing a <video> source, not a <picture> source. So, we have to add the following extra lines:

It's also important to note that the polyfill is Javascript, so a browser that has Javascript disabled will only display the fallback image. These are minor issues, but I found things get more complicated.

Because browsers which rely on this Javascript polyfill don't natively understand the responsive image methods we're using, the browser will first render the DOM, then it will download embedded media, such as images and Javascript, then it will parse the Javascript. This means that browsers using this polyfill will download the fallback image in your img src attribute prior to executing the Javascript that loads the responsive image. This means that our mobile browsers that don't support our chosen responsive image solution (such as Safari!) on their bandwidth-limited connections are being forced to download two images for every responsive image we want to show (the fallback when the DOM renders and the image we want to show when the Javascript runs)! You may even see the fallback image briefly before it flashes to the correct image. That's bad for our users, and potentially bad for our search engine ranking because we're making our page load times unnecessarily long.

To account for this, the Picturefill creators suggest nixing the src attribute from our <img> tags entirely and just using <srcset> without a fallback, because Picturefill is our fallback. If you're using <picture>, you can still mix-and-match as mentioned above, making your <img> fallback a single srcset. This markup is the standard in Picturefill's current examples. What this means is that browsers that don't natively understand responsive images won't see any picture when the DOM loads and will only load an image once Picturefill runs. But wait, didn't we say that Safari understands img srcset, but not <picture>? Ah, yes, that means that Safari will still load the fallback srcset picture during DOM rendering if you're using a <picture> element for your responsive image.

So, to keep Safari from loading a fallback image, we'd have to not provide a fallback image at all! That is, an <img> tag with no srcor srcset, (aka, the "mutilated img pattern"). That way, all browsers will only load the images they natively support or the one suggested by the polyfill. However, this has other drawbacks. Obviously, browsers that have Javascript disabled now have no fallback image to load and won't show anything. Also, very old versions of Android (2.3 and earlier) or IE6-IE8 (without another polyfill) won't see the <source> tag at all and rely on the srcset fallback. However, these browsers have very low adoption rates at this time. Perhaps most critically for some of you, as it was for my project, though, is that search engines won't pick up any images that don't have a src fallback. If making sure your images show up in search results is important to you, you'll probably want to provide a good fallback image.

For my project, I opted to keep the src attribute in all my responsive <img> tags, mainly for the last reason. The drawback of forcing double downloads on Safari and Internet Explorer for the foreseeable future didn't outweigh the drawback of our beautiful images not showing up in Google Image search results. Another thing to consider is that the Picturefill team is aware of the issue and plan to address it, if possible, in Picturefill 3.0, which is currently in alpha status. Whether Picturefill fixes it before Safari and IE add native responsive support shouldn't matter, though, because as long as I'm writing my code following the responsive image standard, whichever one gets there first, everything should work the way it's supposed to. Now, we nearly have the complete picture for handling responsive images today.

Photo by Justus Hayes