rowid,title,contents,year,author,author_slug,published,url,topic 241,Jank-Free Image Loads,"There are a few fundamental problems with embedding images in pages of hypertext; perhaps chief among them is this: text is very light and loads rather fast; images are much heavier and arrive much later. Consequently, millions (billions?) of times a day, a hapless Web surfer will start reading some text on a page, and then — Your browser doesn’t support HTML5 video. Here is a link to the video instead. — oops! — an image pops in above it, pushing said text down the page, and our poor reader loses their place. By default, partially-loaded pages have the user experience of a slippery fish, or spilled jar of jumping beans. For the rest of this article, I shall call that jarring, no-good jumpiness by its name: jank. And I’ll chart a path into a jank-free future – one in which it’s easy and natural to author elements that load like this: Your browser doesn’t support HTML5 video. Here is a link to the video instead. Jank is a very old problem, and there is a very old solution to it: the width and height attributes on . The idea is: if we stick an image’s dimensions right into the HTML, browsers can know those dimensions before the image loads, and reserve some space on the layout for it so that nothing gets bumped down the page when the image finally arrives. width Specifies the intended width of the image in pixels. When given together with the height, this allows user agents to reserve screen space for the image before the image data has arrived over the network. —The HTML 3.2 Specification, published on January 14 1997 Unfortunately for us, when width and height were first spec’d and implemented, layouts were largely fixed and images were usually only intended to render at their fixed, actual dimensions. When image sizing gets fluid, width and height get weird: See the Pen fluid width + fixed height = distortion by Eric Portis (@eeeps) on CodePen. width and height are too rigid for the responsive world. What we need, and have needed for a very long time, is a way to specify fixed aspect ratios, to pair with our fluid widths. I have good news, bad news, and great news. The good news is, there are ways to do this, now, that work in every browser. Responsible sites, and responsible developers, go through the effort to do them. The bad news is that these techniques are all terrible, cumbersome hacks. They’re difficult to remember, difficult to understand, and they can interact with other pieces of CSS in unexpected ways. So, the great news: there are two on-the-horizon web platform features that are trying to make no-jank, fixed-aspect-ratio, fluid-width images a natural part of the web platform. aspect-ratio in CSS The first proposed feature? An aspect-ratio property in CSS! This would allow us to write CSS like this: img { width: 100%; } .thumb { aspect-ratio: 1/1; } .hero { aspect-ratio: 16/9; } This’ll work wonders when we need to set aspect ratios for whole classes of images, which are all sized to fit within pre-defined layout slots, like the .thumb and .hero images, above. Alas, the harder problem, in my experience, is not images with known-ahead-of-time aspect ratios. It’s images – possibly user generated images – that can have any aspect ratio. The really tricky problem is unknown-when-you’re-writing-your-CSS aspect ratios that can vary per-image. Using aspect-ratio to reserve space for images like this requires inline styles: And inline styles give me the heebie-jeebies! As a web developer of a certain age, I have a tiny man in a blue beanie permanently embedded deep within my hindbrain, who cries out in agony whenever I author a style="""" attribute. And you know what? The old man has a point! By sticking super-high-specificity inline styles in my content, I’m cutting off my, (or anyone else’s) ability to change those aspect ratios, for whatever reason, later. How might we specify aspect ratios at a lower level? How might we give browsers information about an image’s dimensions, without giving them explicit instructions about how to style it? I’ll tell you: we could give browsers the intrinsic aspect ratio of the image in our HTML, rather than specifying an extrinsic aspect ratio! A brief note on intrinsic and extrinsic sizing What do I mean by “intrinsic” and “extrinsic?” The intrinsic size of an image is, put simply, how big it’d be if you plopped it onto a page and applied no CSS to it whatsoever. An 800×600 image has an intrinsic width of 800px. The extrinsic size of an image, then, is how large it ends up after CSS has been applied. Stick a width: 300px rule on that same 800×600 image, and its intrinsic size (accessible via the Image.naturalWidth property, in JavaScript) doesn’t change: its intrinsic size is still 800px. But this image now has an extrinsic size (accessible via Image.clientWidth) of 300px. It surprised me to learn this year that height and width are interpreted as presentational hints and that they end up setting extrinsic dimensions (albeit ones that, unlike inline styles, have absolutely no specificity). CSS aspect-ratio lets us avoid setting extrinsic heights and widths – and instead lets us give images (or anything else) an extrinsic aspect ratio, so that as soon as we set one dimension (possibly to a fluid width, like 100%!), the other dimension is set automatically in relation to it. The last tool I’m going to talk about gets us out of the extrinsic sizing game all together — which, I think, is only appropriate for a feature that we’re going to be using in HTML. intrinsicsize in HTML The proposed intrinsicsize attribute will let you do this: That tells the browser, “hey, this image.jpg that I’m using here – I know you haven’t loaded it yet but I’m just going to let you know right away that it’s going to have an intrinsic size of 800×600.” This gives the browser enough information to reserve space on the layout for the image, and ensures that any and all extrinsic sizing instructions, specified in our CSS, will layer cleanly on top of this, the image’s intrinsic size. You may ask (I did!): wait, what if my references multiple resources, which all have different intrinsic sizes? Well, if you’re using srcset, intrinsicsize is a bit of a misnomer – what the attribute will do then, is specify an intrinsic aspect ratio: In the future (and behind the “Experimental Web Platform Features” flag right now, in Chrome 71+), asking this image for its .naturalWidth would not return 3 – it will return whatever 75vw is, given the current viewport width. And Image.naturalHeight will return that width, divided by the intrinsic aspect ratio: 3/2. Can’t wait I seem to have gotten myself into the weeds a bit. Sizing on the web is complicated! Don’t let all of these details bury the big takeaway here: sometime soon (🤞 2019‽ 🤞), we’ll be able to toss our terrible aspect-ratio hacks into the dustbin of history, get in the habit of setting aspect-ratios in CSS and/or intrinsicsizes in HTML, and surf a less-frustrating, more-performant, less-janky web. I can’t wait!",2018,Eric Portis,ericportis,2018-12-21T00:00:00+00:00,https://24ways.org/2018/jank-free-image-loads/,code 242,Creating My First Chrome Extension,"Writing a Chrome Extension isn’t as scary at it seems! Not too long ago, I used a Chrome extension called 20 Cubed. I’m far-sighted, and being a software engineer makes it difficult to maintain distance vision. So I used 20 Cubed to remind myself to look away from my screen and rest my eyes. I loved its simple interface and design. I loved it so much, I often forgot to turn it off in the middle of presentations, where it would take over my entire screen. Oops. Unfortunately, the developer stopped updating the extension and removed it from Chrome’s extension library. I was so sad. None of the other eye rest extensions out there matched my design aesthetic, so I decided to create my own! Want to do the same? Fortunately, Google has some respectable documentation on how to create an extension. And remember, Chrome extensions are just HTML, CSS, and JavaScript. You can add libraries and frameworks, or you can just code the “old-fashioned” way. Sky’s the limit! Setup But first, some things you’ll need to know about before getting started: Callbacks Timeouts Chrome Dev Tools Developing with Chrome extension methods requires a lot of callbacks. If you’ve never experienced the joy of callback hell, creating a Chrome extension will introduce you to this concept. However, things can get confusing pretty quickly. I’d highly recommend brushing up on that subject before getting started. Hyperbole and a Half Timeouts and Intervals are another thing you might want to brush up on. While creating this extension, I didn’t consider the fact that I’d be juggling three timers. And I probably would’ve saved time organizing those and reading up on the Chrome extension Alarms documentation beforehand. But more on that in a bit. On the note of organization, abstraction is important! You might have any combination of the following: The Chrome extension options page The popup from the Chrome Menu The windows or tabs you create The background scripts And that can get unwieldy. You might also edit the existing tabs or windows in the browser, which you’ll probably want as a separate script too. Note that this tutorial only covers creating your own customized window rather than editing existing windows or tabs. Alright, now that you know all that up front, let’s get going! Documentation TL;DR READ THE DOCS. A few things to get started: Read Google’s primer on browser extensions Have a look at their Getting started tutorial Check out their overview on Chrome Extensions This overview discusses the Chrome extension files, architecture, APIs, and communication between pages. Funnily enough, I only discovered the Overview page after creating my extension. The manifest.json file gives the browser information about the extension, including general information, where to find your extension files and icons, and API permissions required. Here’s what my manifest.json looked like, for example: https://github.com/jennz0r/eye-rest/blob/master/manifest.json Because I’m a visual learner, I found the images that describe the extension’s architecture most helpful. To clarify this diagram, the background.js file is the extension’s event handler. It’s constantly listening for browser events, which you’ll feed to it using the Chrome Extension API. Google says that an effective background script is only loaded when it is needed and unloaded when it goes idle. The Popup is the little window that appears when you click on an extension’s icon in the Chrome Menu. It consists of markup and scripts, and you can tell the browser where to find it in the manifest.json under page_action: { ""default_popup"": FILE_NAME_HERE }. The Options page is exactly as it says. This displays customizable options only visible to the user when they either right-click on the Chrome menu and choose “Options” under an extension. This also consists of markup and scripts, and you can tell the browser where to find it in the manifest.json under options_page: FILE_NAME_HERE. Content scripts are any scripts that will interact with any web windows or tabs that the user has open. These scripts will also interact with any tabs or windows opened by your extension. Debugging A quick note: don’t forget the debugging tutorial! Just like any other Chrome window, every piece of an extension has an inspector and dev tools. If (read: when) you run into errors (as I did), you’re likely to have several inspector windows open – one for the background script, one for the popup, one for the options, and one for the window or tab the extension is interacting with. For example, I kept seeing the error “This request exceeds the MAX_WRITE_OPERATIONS_PER_HOUR quota.” Well, it turns out there are limitations on how often you can sync stored information. Another error I kept seeing was “Alarm delay is less than minimum of 1 minutes. In released .crx, alarm “ALARM_NAME_HERE” will fire in approximately 1 minutes”. Well, it turns out there are minimum interval times for alarms. Chrome Extension creation definitely benefits from debugging skills. Especially with callbacks and listeners, good old fashioned console.log can really help! Me adding a ton of `console.log`s while trying to debug my alarms. Eye Rest Functionality Ok, so what is the extension I created? Again, it’s a way to rest your eyes every twenty minutes for twenty seconds. So, the basic functionality should look like the following: If the extension is running AND If the user has not clicked Pause in the Popup HTML AND If the counter in the Popup HTML is down to 00:00 THEN Open a new window with Timer HTML AND Start a 20 sec countdown in Timer HTML AND Reset the Popup HTML counter to 20:00 If the Timer HTML is down to 0 sec THEN Close that window. Rinse. Repeat. Sounds simple enough, but wow, these timers became convoluted! Of all the Chrome extensions I decided to create, I decided to make one that’s heavily dependent on time, intervals, and having those in sync with each other. In other words, I made this unnecessarily complicated and didn’t realize until I started coding. For visual reference of my confusion, check out the GitHub repository for Eye Rest. (And yes, it’s a pun.) API Now let’s discuss the APIs that I used to build this extension. Alarms What even are alarms? I didn’t know either. Alarms are basically Chrome’s setTimeout and setInterval. They exist because, as Google says… DOM-based timers, such as window.setTimeout() or window.setInterval(), are not honored in non-persistent background scripts if they trigger when the event page is dormant. For more information, check out this background migration doc. One interesting note about alarms in Chrome extensions is that they are persistent. Garbage collection with Chrome extension alarms seems unreliable at best. I didn’t have much luck using the clearAll method to remove alarms I created on previous extension loads or installs. A workaround (read: hack) is to specify a unique alarm name every time your extension is loaded and clearing any other alarms without that unique name. Background Scripts For Eye Rest, I have two background scripts. One is my actual initializer and event listener, and the other is a helpers file. I wanted to share a couple of functions between my Background and Popup scripts. Specifically, the clearAndCreateAlarm function. I wanted my background script to clear any existing alarms, create a new alarm, and add remaining time until the next alarm to local storage immediately upon extension load. To make the function available to the Background script, I added helpers.js as the first item under background > scripts in my manifest.json. I also wanted my Popup script to do the same things when the user has unpaused the extension’s functionality. To make the function available to the Popup script, I just include the helpers script in the Popup HTML file. Other APIs Windows I use the Windows API to create the Timer window when the time of my alarm is up. The window creation is initiated by my Background script. One day, while coding late into the evening, I found it very confusing that the window.create method included url as an option. I assumed it was meant to be an external web address. A friend pondered that there must be an option to specify the window’s HTML. Until then, it hadn’t dawned on me that the url could be relative. Duh. I was tired! I pass the timer.html as the url option, as well as type, size, position, and other visual options. Storage Maybe you want to pass information back and forth between the Background script and your Popup script? You can do that using Chrome or local storage. One benefit of using local storage over Chrome’s storage is avoiding quotas and write operation maximums. I wanted to pass the time at which the latest alarm was set, the time to the next alarm, and whether or not the timer is paused between the Background and Popup scripts. Because the countdown should change every second, it’s quite complicated and requires lots of writes. That’s why I went with the user’s local storage. You can see me getting and setting those variables in my Background, Helper, and Popup scripts. Just search for date, nextAlarmTime, and isPaused. Declarative Content The Declarative Content API allows you to show your extension’s page action based on several type of matches, without needing to take a host permission or inject a content script. So you’ll need this to get your extension to work in the browser! You can see me set this in my Background script. Because I want my extension’s popup to appear on every page one is browsing, I leave the page matchers empty. There are many more APIs for Chrome apps and extensions, so make sure to surf around and see what features are available! The Extension Here’s what my original Popup looked like before I added styles. And here’s what it looks like with new styles. I guess I’m going for a Nickelodeon feel. And here’s the Timer window and Popup together! Publishing Publishing is a cinch. You just zip up your files, create a new or use an existing Google Developer account, upload the files, add some details, and pay a one time $5 fee. That’s all! Then your extension will be available on the Chrome extension store! Neato :D My extension is now available for you to install. Conclusion I thought creating a time based Chrome Extension would be quick and easy. I was wrong. It was more complicated than I thought! But it’s definitely achievable with some time, persistence, and good ole Google searches. Eventually, I’d like to add more interactive elements to Eye Rest. For example, hitting the YouTube API to grab a silly or cute video as a reward for looking away during the 20 sec countdown and not closing the timer window. This harkens back to one of my first web projects, Toothtimer, from 2012. Or maybe a way to change the background colors of the Timer and Popup! Either way, with Eye Rest’s framework built out, I’m feeling fearless about future feature adds! Building this Chrome extension took some broken nails, achy shoulders, and tired eyes, but now Eye Rest can tell me to give my eyes a break every 20 minutes.",2018,Jennifer Wong,jenniferwong,2018-12-05T00:00:00+00:00,https://24ways.org/2018/my-first-chrome-extension/,code 243,Researching a Property in the CSS Specifications,"I frequently joke that I’m “reading the specs so you don’t have to”, as I unpack some detail of a CSS spec in a post on my blog, some documentation for MDN, or an article on Smashing Magazine. However waiting for someone like me to write an article about something is a pretty slow way to get the information you need. Sometimes people like me get things wrong, or specifications change after we write a tutorial. What if you could just look it up yourself? That’s what you get when you learn to read the CSS specifications, and in this article my aim is to give you the basic details you need to grab quick information about any CSS property detailed in the CSS specs. Where are the CSS Specifications? The easiest way to see all of the CSS specs is to take a look at the Current Work page in the CSS section of the W3C Website. Here you can see all of the specifications listed, the level they are at and their status. There is also a link to the specification from this page. I explained CSS Levels in my article Why there is no CSS 4. Who are the specifications for? CSS specifications are for everyone who uses CSS. You might be a browser engineer - referred to as an implementor - needing to know how to implement a feature, or a web developer - referred to as an author - wanting to know how to use the feature. The fact that both parties are looking at the same document hopefully means that what the browser displays is what the web developer expected. Which version of a spec should I look at? There are a couple of places you might want to look. Each published spec will have the latest published version, which will have TR in the URL and can be accessed without a date (which is always the newest version) or at a date, which will be the date of that publication. If I’m referring to a particular Working Draft in an article I’ll typically link to the dated version. That way if the information changes it is possible for someone to see where I got the information from at the time of writing. If you want the very latest additions and changes to the spec, then the Editor’s Draft is the place to look. This is the version of the spec that the editors are committing changes to. If I make a change to the Multicol spec and push it to GitHub, within a few minutes that will be live in the Editor’s Draft. So it is possible there are errors, bits of text that we are still working out and so on. The Editor’s Draft however is definitely the place to look if you are wanting to raise an issue on a spec, as it may be that the issue you are about to raise is already fixed. If you are especially keen on seeing updates to specifications keep an eye on https://drafts.csswg.org/ as this is a list of drafts, along with the date they were last updated. How to approach a spec The first thing to understand is that most CSS Specifications start with the most straightforward information, and get progressively further into the weeds. For an author the initial examples and explanations are likely to be of interest, and then the property definitions and examples. Therefore, if you are looking at a vast spec, know that you probably won’t need to read all the way to the bottom, or read every section in detail. The second thing that is useful to know about modern CSS specifications is how modularized they are. It really never is a case of finding everything you need in a single document. If we tried to do that, there would be a lot of repetition and likely inconsistency between specs. There are some key specifications that many other specifications draw on, such as: Values and Units Intrinsic and Extrinsic Sizing Box Alignment When something is defined in another specification the spec you are reading will link to it, so it is worth opening that other spec in a new tab in order that you can refer back to it as you explore. Researching your property As an example we will take a look at the property grid-auto-rows, this property defines row tracks in the implicit grid when using CSS Grid Layout. The first thing you will need to do is find out which specification defines this property. You might already know which spec the property is part of, and therefore you could go directly to the spec and search using your browser or look in the navigation for the spec to find it. Alternatively, you could take a look at the CSS Property Index, which is an automatically generated list of CSS Properties. Clicking on a property will take you to the TR version of the spec, the latest published draft, and the definition of that property in it. This definition begins with a panel detailing the syntax of this property. For grid-auto-rows, you can see that it is listed along with grid-auto-columns as these two properties are essentially identical. They take the same values and work in the same way, one for rows and the other for columns. Value For value we can see that the property accepts a value . The next thing to do is to find out what that actually means, clicking will take you to where it is defined in the Grid spec. The value is defined as accepting various values: minmax( , ) fit-content( We need to head down the rabbit hole to find out what each of these mean. From here we essentially go down line by line until we have unpacked the value of track-size. is defined just below as: min-content max-content auto So these are all things that would be valid to use as a value for grid-auto-rows. The first value of is something you will see in many specifications as a value. It means that you can use a length unit - for example px or em - or a percentage. Some properties only accept a in which case you know that you cannot use a percentage as the value. This means that you could have grid-auto-rows with any of the following values. grid-auto-rows: 100px; grid-auto-rows: 1em; grid-auto-rows: 30%; When using percentages, it is important to know what it is a percentage of. As a percentage has to resolve from something. There is text in the spec which explains how column and row percentages work. “ values are relative to the inline size of the grid container in column grid tracks, and the block size of the grid container in row grid tracks.” This means that in a horizontal writing mode such as when using English, a percentage when used as a track-size in grid-auto-columns would be a percentage of the width of the grid, and a percentage in grid-auto-rows a percentage of the height of the grid. The second value of is also defined here, as “A non-negative dimension with the unit fr specifying the track’s flex factor.” This is the fr unit, and the spec links to a fuller definition of fr as this unit is only used in Grid Layout so it is therefore defined in the grid spec. We now know that a valid value would be: grid-auto-rows: 1fr; There is some useful information about the fr unit in this part of the spec. It is noted that the fr unit has an automatic minimum. This means that 1fr is really minmax(auto, 1fr). This is why having a number of tracks all at 1fr does not mean that all are equal sized, as a larger item in any of the tracks would have a large auto size and therefore would be larger after spare space had been distributed. We then have min-content and max-content. These keywords can be used for track sizing and the specification defines what they mean in the context of sizing a track, representing the min and max-sizing contributions of the grid tracks. You will see that there are various terms linked in the definition, so if you do not know what these mean you can follow them to find out. For example the spec links max-content contribution to the CSS Intrinsic and Extrinsic Sizing specification. This is one of those specs which is drawn on by many other specifications. If we follow that link we can read the definition there and follow further links to understand what each term means. The more that you read specifications the more these terms will become familiar to you. Just like learning a foreign language, at first you feel like you have to look up every little thing. After a while you remember the vocabulary. We can now add min-content and max-content to our available values. grid-auto-rows: min-content; grid-auto-rows: max-content; The final item in our list is auto. If you are familiar with using Grid Layout, then you are probably aware that an auto sized track for will grow to fit the content used. There is an interesting note here in the spec detailing that auto sized rows will stretch to fill the grid container if there is extra space and align-content or justify-content have a value of stretch. As stretch is the default value, that means these tracks stretch by default. Tracks using other types of length will not behave like this. grid-auto-rows: auto; So, this was the list for , the next possible value is minmax( , ). So this is telling us that we can use minmax() as a value, the final (max) value will be and we have already unpacked all of the allowable values there. The first value (min) is detailed as an . If we look at the values for this, we discover that they are the same as , minus the value: min-content max-content auto We already know what all of these do, so we can add possible minmax() values to our list of values for . grid-auto-rows: minmax(100px, 200px); grid-auto-rows: minmax(20%, 1fr); grid-auto-rows: minmax(1em, auto); grid-auto-rows: minmax(min-content, max-content); Finally we can use fit-content( . We can see that fit-content takes a value of which we already know to be either a length unit, or a percentage. The spec details how fit-content is worked out, and it essentially allows a track which acts as if you had used the max-content keyword, however the track stops growing when it hits the length passed to it. grid-auto-rows: fit-content(200px); grid-auto-rows: fit-content(20%); Those are all of our possible values, and to round things off, check again at the initial value, you can see it has a little + sign next to it, click that and you will be taken to the CSS Values and Units module to find that, “A plus (+) indicates that the preceding type, word, or group occurs one or more times.” This means that we can pass a single track size to grid-auto-rows or multiple track sizes as a space separated list. Below the box is an explanation of what happens if you pass in more than one track size: “If multiple track sizes are given, the pattern is repeated as necessary to find the size of the implicit tracks. The first implicit grid track after the explicit grid receives the first specified size, and so on forwards; and the last implicit grid track before the explicit grid receives the last specified size, and so on backwards.” Therefore with the following CSS, if five implicit rows were needed they would be as follows: 100px 1fr auto 100px 1fr .grid { display: grid; grid-auto-rows: 100px 1fr auto; } Initial We can now move to the next line in the box, and you’ll be glad to know that it isn’t going to require as much unpacking! This simply defines the initial value for grid-auto-rows. If you do not specify anything, created rows will be auto sized. All CSS properties have an initial value that they will use if they are invoked as part of the usage of the specification they are in, but you do not set a value for them. In the case of grid-auto-rows it is used whenever rows are created in the implicit grid, so it needs to have a value to be used even if you do not set one. Applies to This line tells us what this property is used for. Some properties are used in multiple places. For example if you look at the definition for justify-content in the Box Alignment specification you can see it is used in multicol containers, flex containers, and grid containers. In our case the property only applies for grid containers. Inherited This tells us if the property can be inherited from a parent element if it is not set. In the case of grid-auto-rows it is not inherited. A property such as color is inherited, so you do not need to set it on each element. Percentages Are percentages allowed for this property, and if so how are they calculated. In this case we are referred to the definition for grid-template-columns and grid-template-rows where we discover that the percentage is from the corresponding dimension of the content area. Media This defines the media group that the property belongs to. In this case visual. Computed Value This details how the value is resolved. The grid-auto-rows property again refers to track sizing as defined for grid-template-columns and grid-template-rows, which tells us the computed value is as specified with lengths made absolute. Canonical Order If you have a property–generally a shorthand property–which takes multiple values in a set order, then those values need to be serialized in the order detailed in the grammar for that property. In general you don’t need to worry about this value in the table. Animation Type This details whether the property can be animated, and if so what type of animation. This is useful if you are trying to animate something and not getting the result that you expect. Note that just because something is listed in the spec as animatable does not mean that browsers will have implemented animation for that property yet! That’s (mostly) it! Sometimes the property will have additional examples - there is one underneath the table for grid-auto-rows. These are worth looking at as they will highlight usage of the property that the spec editor has felt could use an example. There may also be some additional text explaining anythign specific to this property. In selecting grid-auto-rows I chose a fairly complex property in terms of the work we needed to do to unpack the value. Many properties are far simpler than this. However ultimately, even when you come across a complex value, it really is just a case of stepping through the definitions until you come to the bottom of the rabbit hole. Being able to work out what is valid for each property is incredibly useful. It means you don’t waste time trying to use a value that doesn’t work for that property. You also may find that there are values you weren’t aware of, that solve problems for you. Further reading Specifications are not designed to be user manuals, and while they often contain examples, these are pretty terse as they need to be clear to demonstrate their particular point. The manual for the Web Platform is MDN Web Docs. Pairing reading a specification with the examples and information on an MDN property page such as the one for grid-auto-rows is a really great way to ensure that you have all the information and practical usage examples you might need. You may also find useful: Value Definition Syntax on MDN. The MDN Glossary defines many common terms. Understanding the CSS Property Value Syntax goes into more detail in terms of reading the syntax. How to read W3C Specs - from 2001 but still relevant. I hope this article has gone some way to demystify CSS specifications for you. Even if the specifications are not your preferred first stop to learn about new CSS, being able to go directly to the source and avoid having your understanding filtered by someone else, can be very useful indeed.",2018,Rachel Andrew,rachelandrew,2018-12-14T00:00:00+00:00,https://24ways.org/2018/researching-a-property-in-the-css-specifications/,code 244,It’s Beginning to Look a Lot Like XSSmas,"I dread the office Secret Santa. I have a knack for choosing well-meaning but inappropriate presents, like a bottle of port for a teetotaller, a cheese-tasting experience for a vegan, or heaven forbid, Spurs socks for an Arsenal supporter. Ok, the last one was intentional. It’s the same with gifting code. Once, I made a pattern library for A List Apart which I open sourced, and a few weeks later, a glaring security vulnerability was found in it. My gift was so generous that it enabled unrestricted access to any file on any public-facing server that hosted it. With platforms like GitHub and npm, giving the gift of code is so easy it’s practically a no-brainer. This giant, open source yankee swap helps us do our jobs without starting from scratch with every project. But like any gift-giving, it’s also risky. Vulnerabilities and Open Source Open source code is not inherently more or less vulnerable than closed-source code. What makes it higher risk is that the same piece of code gets reused in lots of places, meaning a hacker can use the same exploit mechanism on the same vulnerable code in different apps. Graph showing the number of open source vulnerabilities published per year, from the State of Open Source Security 2017 In the first 24 ways article this year, Katie referenced a few different types of vulnerability: Cross-site Request Forgery (also known as CSRF) SQL Injection Cross-site Scripting (also known as XSS) There are many more types of vulnerability, and those that live under the same category share similarities. For example, my favourite – is it weird to have a favourite vulnerability? – is Cross Site Scripting (XSS), which allows for the injection of scripts into web pages. This is a really common vulnerability often unwittingly added by developers. OWASP (the Open Web Application Security Project) wrote a great article about how to prevent opening the door to XSS attacks – share it generously with your colleagues. Most vulnerabilities like this are not added intentionally – they’re doors left ajar due to the way something has been scripted, like the over-generous code in my pattern library. Others, though, are added intentionally. A few months ago, a hacker, disguised as a helpful elf, offered to take over the maintenance of a popular npm package that had been unmaintained for a couple of years. The owner had moved onto other projects, and was keen to see it continue to be maintained by someone else, so transferred ownership. Fast-forward 3 months, it was discovered that the individual had quietly added a malicious package to the codebase, and the obfuscated code in it had been unwittingly installed onto thousands of apps. The code added was designed to harvest Bitcoin if it was run alongside another application. It was only spotted due to a developer’s curiosity. Another tactic to get developers to unwittingly install malicious packages into their codebase is “typosquatting” – back in August last year, npm reported that a user had been publishing packages with very similar names to popular packages (for example, crossenv instead of cross-env). This is a big wakeup call for open source maintainers. Techniques like this are likely to be used more as the maintenance of open source libraries becomes an increasing burden to their owners. After all, starting a new project often has a greater reward than maintaining an existing one, but remember, an open source library is for life, not just for Christmas. Santa’s on his sleigh If you use open source libraries, chances are that these libraries also use open source libraries. Your app may only have a handful of dependencies, but tucked in the back of that sleigh may be a whole extra sack of dependencies known as deep dependencies (ones that you didn’t directly install, but are dependencies of that dependency), and these can contain vulnerabilities too. Let’s look at the npm package santa as an example. santa has 8 direct dependencies listed on npm. That seems pretty manageable. But that’s just the tip of the iceberg – have a look at the full dependency tree which contains 109 dependencies – more dependencies than there are Christmas puns in this article. Only one of these direct dependencies has a vulnerability (at the time of writing), but there are actually 13 other known vulnerabilities in santa, which have been introduced through its deeper dependencies. Fixing vulnerabilities – the ultimate christmas gift If you’re a maintainer of open source libraries, taking good care of them is the ultimate gift you can give. Keep your dependencies up to date, use a security tool to monitor and alert you when new vulnerabilities are found in your code, and fix or patch them promptly. This will help keep the whole open source ecosystem healthy. When you find out about a new vulnerability, you have some options: Fix the vulnerability via an upgrade You can often fix a vulnerability by upgrading the library to the latest version. Make sure you’re using software that monitors your dependencies for new security issues and lets you know when a fix is ready, otherwise you may be unwittingly using a vulnerable version. Patch the vulnerable code Sometimes, a fix for a vulnerable library isn’t possible. This is often the case when a library is no longer being maintained, or the version of the library being used might be so out of date that upgrading it would cause a breaking change. Patches are bits of code that will fix that particular issue, but won’t change anything else. Switch to a different library If the library you’re using has no fix or patch, you may be better of switching it out for another one, particularly if it looks like it’s being unmaintained. Responsibly disclosing vulnerabilities Knowing how to responsibly disclose vulnerabilities is something I’m ashamed to admit that I didn’t know about before I joined a security company. But it’s so important! On discovering a new vulnerability, a developer has a few options: A malicious developer will exploit that vulnerability for their own gain. A reckless (or inexperienced) developer will disclose that vulnerability to the world without following a responsible disclosure process. This opens the door to an unethical developer exploiting the vulnerability. At Snyk, we monitor social media for mentions of newly found vulnerabilities so we can add them to our database and share fixes before they get exploited. An ethical and aware developer will follow what’s known as a “responsible disclosure process”. They will contact the maintainer of the code privately, allowing reasonable time for them to release a fix for the issue and to give others who use that vulnerable code a chance to fix it too. It’s important to understand this process if you’re a maintainer or contributor of code. It can be daunting when a report comes in, but understanding and following the right steps will help reduce the risk to the people who use that code. So what does responsible disclosure look like? I’ll take Node.js’s security disclosure policy as an example. They ask that all security issues that are found in Node.js are reported there. (There’s a separate process for bug found in third-party npm packages). Once you’ve reported a vulnerability, they promise to acknowledge it within 24 hours, and to give a more detailed response within 48 hours. If they find that the issue is indeed a security bug, they’ll give you regular updates about the progress they’re making towards fixing it. As part of this, they’ll figure out which versions are affected, and prepare fixes for them. They’ll assign the vulnerability a CVE (Common Vulnerabilities and Exposures) ID and decide on an embargo date for public disclosure. On the date of the embargo, they announce the vulnerability in their Node.js security mailing list and deploy fixes to nodejs.org. Tim Kadlec published an in-depth article about responsible disclosures if you’re interested in knowing more. It has some interesting horror stories of what happened when the disclosure process was not followed. Encourage responsible disclosure Add a SECURITY.md file to your project so someone who wants to message you about a vulnerability can do so without having to hunt around for contact details. Last year, Snyk published a State of Open Source Security report that found 79.5% of maintainers do not have a public disclosure policy. Those that did were considerably more likely to get notified privately about a vulnerability – 73% of maintainers who had one had been notified, vs 21% of maintainers who hadn’t published one one. Stats from the State of Open Source Security 2017 Bug bounties Some companies run bug bounties to encourage the responsible disclosure of vulnerabilities. By offering a reward for finding and safely disclosing a vulnerability, it also reduces the enticement of exploiting a vulnerability over reporting it and getting a quick cash reward. Hackerone is a community of ethical hackers who pentest apps that have signed up for the scheme and get paid when they find a new vulnerability. Wordpress is one such participant, and you can see the long list of vulnerabilities that have been disclosed as part of that program. If you don’t have such a bounty, be prepared to get the odd vulnerability extortion email. Scott Helme, who founded securityheaders.com and report-uri.com, wrote a post about some of the requests he gets for a report about a critical vulnerability in exchange for money. On one hand, I want to be as responsible as possible and if my users are at risk then I need to know and patch this issue to protect them. On the other hand this is such irresponsible and unethical behaviour that interacting with this person seems out of the question. A gift worth giving It’s time to brush the dust off those old gifts that we shared and forgot about. Practice good hygiene and run them through your favourite security tool – I’m just a little biased towards Snyk, but as Katie mentioned, there’s also npm audit if you use Node.js, and most source code managers like GitHub and GitLab have basic vulnerability alert capabilities. Stats from the State of Open Source Security 2017 Most importantly, patch or upgrade away those vulnerabilities away, and if you want to share that Christmas spirit, open fixes for your favourite open source projects, too.",2018,Anna Debenham,annadebenham,2018-12-17T00:00:00+00:00,https://24ways.org/2018/its-beginning-to-look-a-lot-like-xssmas/,code 246,Designing Your Site Like It’s 1998,"It’s 20 years to the day since my wife and I started Stuff & Nonsense, our little studio and my outlet for creative ideas on the web. To celebrate this anniversary—and my fourteenth contribution to 24 ways— I’d like to explain how I would’ve developed a design for Planes, Trains and Automobiles, one of my favourite Christmas films. My design for Planes, Trains and Automobiles is fixed at 800px wide. Developing a framework I’ll start by using frames to set up the framework for this new website. Frames are individual pages—one for navigation, the other for my content—pulled together to form a frameset. Space is limited on lower-resolution screens, so by using frames I can ensure my navigation always remains visible. I can include any number of frames inside a element. I add two rows to my ; the first is for my navigation and is 50px tall, the second is for my content and will resize to fill any available space. As I don’t want frame borders or any space between my frames, I set frameborder and framespacing attributes to 0: […] Next I add the source of my two frame documents. I don’t want people to be able to resize or scroll my navigation, so I add the noresize attribute to that frame: I do want links from my navigation to open in the content frame, so I give each a name so I can specify where I want links to open: The framework for this website is simple as it contains only two horizontal rows. Should I need a more complex layout, I can nest as many framesets—and as many individual documents—as I need: Letterbox framesets were common way to deal with multiple screen sizes. In a letterbox, the central frameset had a fixed height and width, while the frames on the top, right, bottom, and left expanded to fill any remaining space. Handling older browsers Sadly not every browser supports frames, so I should send a helpful message to people who use older browsers asking them to upgrade. Happily, I can do that using noframes content: <body> <p>This page uses frames, but your browser doesn’t support them. Please upgrade your browser.</p> </body> Forcing someone back into a frame Sometimes, someone may follow a link to a page from a portal or search engine, or they might attempt to open it in a new window or tab. If that page properly belongs inside a , people could easily miss out on other parts of a design. This short script will prevent this happening and because it’s vanilla Javascript, it doesn’t require a library such as jQuery: Laying out my page Before starting my layout, I add a few basic background and colour styles. I must include these attributes in every page on my website: I want absolute control over how people experience my design and don’t want to allow it to stretch, so I first need a which limits the width of my layout to 800px. The align attribute will keep this
in the centre of someone’s screen:
[…]
Although they were developed for displaying tabular information, the cells and rows which make up the element make it ideal for the precise implementation of a design. I need several tables—often nested inside each other—to implement my design. These include tables for a banner and three rows of content:
[…]
[…]
[…]
[…]
The width of the first table—used for my banner—is fixed to match the logo it contains. As I don’t need borders, padding, or spacing between these cells, I use attributes to remove them:
The next table—which contains the largest image, introduction, and a call-to-action—is one of the most complex parts of my design, so I need to ensure its layout is pixel perfect. To do that I add an extra row at the top of this table and fill each of its cells with tiny transparent images: The height and width of these “shims” or “spacers” is only 1px but they will stretch to any size without increasing their weight on the page. This makes them perfect for performant website development. For the hero of this design, I splice up the large image into three separate files and apply each slice as a background to the table cells. I also match the height of those cells to the background images:   […]   I use tables and spacer images throughout the rest of this design to lay out the various types of content with perfect precision. For example, to add a single-pixel border around my two columns of content, I first apply a blue background to an outer table along with 1px of cellspacing, then simply nest an inner table—this time with a white background—inside it:
[…]
Adding details Tables are fabulous tools for laying out a page, but they’re also useful for implementing details on those pages. I can use a table to add a gradient background, rounded corners, and a shadow to the button which forms my “Buy the DVD” call-to-action. First, I splice my button graphic into three slices; two fixed-width rounded ends, plus a narrow gradient which stretches and makes this button responsive. Then, I add those images as backgrounds and use spacers to perfectly size my button:
Buy the DVD
I use those same elements to add details to headlines and lists too. Adding a “bullet” to each item in a list needs only two additional table cells, a circular graphic, and a spacer:
    Directed by John Hughes
Implementing a typographic hierarchy So far I’ve explained how to use frames, tables, and spacers to develop a layout for my content, but what about styling that content? I use elements to change the typeface from the browser’s default to any font installed on someone’s device: Planes, Trains and Automobiles is a comedy film […] To adjust the size of those fonts, I use the size attribute and a value between the smallest (1) and the largest (7) where 3 is the browser’s default. I use a size of 4 for this headline and 2 for the text which follows: Steve Martin An American actor, comedian, writer, producer, and musician. When I need to change the typeface, perhaps from a sans-serif like Arial to a serif like Times New Roman, I must change the value of the face attribute on every element on all pages on my website. NB: I use as many
elements as needed to create space between headlines and paragraphs. View the final result (and especially the source.) My modern day design for Planes, Trains and Automobiles. I can imagine many people reading this and thinking “This is terrible advice because we don’t develop websites like this in 2018.” That’s true. We have the ability to embed any number of web fonts into our products and websites and have far more control over type features, leading, ligatures, and sizes: font-variant-caps: titling-caps; font-variant-ligatures: common-ligatures; font-variant-numeric: oldstyle-nums; Grid has simplified the implementation of even the most complex compound grid down to just a few lines of CSS: body { display: grid; grid-template-columns: 3fr 1fr 2fr 2fr 1fr 3fr; grid-template-rows: auto; grid-column-gap: 2vw; grid-row-gap: 1vh; } Flexbox has made it easy to develop flexible components such as navigation links: nav ul { display: flex; } nav li { flex: 1; } Just one line of CSS can create multiple columns of fluid type: main { column-width: 12em; } CSS Shapes enable text to flow around irregular shapes including polygons: [src*=""main-img""] { float: left; shape-outside: polygon(…); } Today, we wouldn’t dream of using images and a table to add a gradient, rounded corners, and a shadow to a button or link, preferring instead: .btn { background: linear-gradient(#8B1212, #DD3A3C); border-radius: 1em; box-shadow: 0 2px 4px 0 rgba(0,0,0,0.50), inset 0 -1px 1px 0 rgba(0,0,0,0.50); } CSS Custom Properties, feature and media queries, filters, pseudo-elements, and SVG; the list of advances in HTML, CSS, and other technologies goes on. So does our understanding of how best to use them by separating content, structure, presentation, and behaviour. As 2018 draws to a close, we’re certain we know how to design and develop products and websites better than we did at the end of 1998. Strange as it might seem looking back, in 1998 we were also certain our techniques and technologies were the best for the job. That’s why it’s dangerous to believe with absolute certainty that the frameworks and tools we increasingly rely on today—tools like Bootstrap, Bower, and Brunch, Grunt, Gulp, Node, Require, React, and Sass—will be any more relevant in the future than elements, frames, layout tables, and spacer images are today. I have no prediction for what the web will be like twenty years from now. However, I want to believe we’ll build on what we’ve learned during these past two decades about the importance of accessibility, flexibility, and usability, and that the mistakes we made while infatuated by technologies won’t be repeated. Head over to my website if you’d like to read about how I’d implement my design for ‘Planes, Trains and Automobiles’ today.",2018,Andy Clarke,andyclarke,2018-12-23T00:00:00+00:00,https://24ways.org/2018/designing-your-site-like-its-1998/,code 247,Managing Flow and Rhythm with CSS Custom Properties,"An important part of designing user interfaces is creating consistent vertical rhythm between elements. Creating consistent, predictable space doesn’t just make your web pages and views look better, but it can also improve the scan-ability. Browsers ship with default CSS and these styles often create consistent rhythm for flow elements out of the box. The problem is though that we often reset these styles with a reset. Elements such as
and
also have no default margin or padding associated with them. I’ve tried all sorts of weird and wonderful techniques to find a balance between using inherited CSS while also levelling the playing field for component driven front-ends with very little success. This experimentation is how I landed on the flow utility, though and I’m going to show you how it works. Let’s dive in! The Flow utility With the ever-growing number of folks working with component libraries and design systems, we could benefit from a utility that creates space for us, only when it’s appropriate to do so. The problem with my previous attempts at fixing this is that the spacing values were very rigid. That’s fine for 90% of contexts, but sometimes, it’s handy to be able to tweak the values based on the exact context of your component. This is where CSS Custom Properties come in handy. The code .flow { --flow-space: 1em; } .flow > * + * { margin-top: var(--flow-space); } What this code does is enable you to add a class of flow to an element which will then add margin-top to sibling elements within that element. We use the lobotomised owl selector to select these siblings. This approach enables an almost anonymous and automatic system which is ideal for component library based front-ends where components probably don’t have any idea what surrounds them. The other important part of this utility is the usage of the --flow-space custom property. We define it in the .flow component and each element within it will be spaced by --flow-space, by default. The beauty about setting this as a custom property is that custom properties also participate in the cascade, so we can utilise specificity to change it if we need it. Pretty cool, right? Let’s look at some examples. A basic example See the Pen CSS Flow Utility: Basic implementation by Andy Bell (@hankchizljaw) on CodePen. https://codepen.io/hankchizljaw/pen/LXqerj What we’ve got in this example is some basic HTML content that has a class of flow on the parent article element. Because there’s a very heavy-handed reset added as a dependency, all of the content would have been squished together without the flow utility. Because our --flow-space custom property is set to 1em, the space between elements is 1X the font size of the element in question. This means that a

in this context has a calculated margin-top value of 28.8px, because it has an assigned font size of 1.8rem. If we were to globally change the --flow-space value to 1.1em for example, we’d affect everything because margin values would be calculated as 1.1X the font size. This example looks great because using font size as the basis of rhythm works really well. What if we wanted to to tweak certain elements within this article, though? See the Pen CSS Flow Utility: Tweaked Basic implementation by Andy Bell (@hankchizljaw) on CodePen. https://codepen.io/hankchizljaw/pen/qQgxaY I like lots of whitespace with my article layouts, so the 1em space isn’t going to cut it for all elements. I like to provide plenty of space between headed sections, so I increase the --flow-space in these instances: h2 { --flow-space: 3rem; } Notice also how I also switch over to using rem units? I want to make sure that these overrides are always based on the root font size. This is a personal preference of mine and you can use whatever units you want. Just be aware that it’s better for accessibility to use flexible units like em, rem and %, so that a user’s font size preferences are honoured. A more advanced example Although the flow utility is super useful for a plethora of contexts, it really shines when working with a few unrelated components. Instead of having to write specific layout CSS just for your particular context, you can use flow and --flow-space to create predictable and contextual space. See the Pen CSS Flow Utility: Unrelated components by Andy Bell (@hankchizljaw) on CodePen. https://codepen.io/hankchizljaw/pen/ZmPGyL In this example, we’ve got ourselves a little prototype layout that features a media element, followed by a grid of features. By using flow, it was really quick and easy to generate space between those two main elements. It was also easy to create space within the components. For example, I added it to the .media__content element, so that the article’s content would space itself:
...
Something to remember though: the custom properties cascade in the same way that other CSS values do, so you’ve got to keep that in mind. We’ve got a great example of that in this example where because we’ve got the flow utility on our .features component, which has a --flow-space override: the child elements of .features will inherit that value, so we’ve had to set another value on the .features__list element. “But what about old browsers?”, I hear you cry We’re using CSS Custom Properties that at the time of writing, have about 88% support. One thing we can do to remedy the other 12% of browsers is to set a default, traditional margin-top value of 1em, so it calculates itself based on the element’s font-size: .flow { --flow-space: 1em; } .flow > * + * { margin-top: 1em; margin-top: var(--flow-space); } Thanks to the cascading and declarative nature of CSS, we can set that default margin-top value and then immediately set it to use the custom property instead. Browsers that understand Custom Properties will automatically apply them—those that don’t will ignore them. Yay for the cascade and progressive enhancement! Wrapping up This tiny little utility can bring great power for when you want to consistently space elements, vertically. It also—thanks to the power of the modern web—allows us to create contextual overrides without creating modifier classes or shame CSS. If you’ve got other methods of doing this sort of work, please let me know on Twitter. I’d love to see what you’re working on!",2018,Andy Bell,andybell,2018-12-07T00:00:00+00:00,https://24ways.org/2018/managing-flow-and-rhythm-with-css-custom-properties/,code 249,Fast Autocomplete Search for Your Website,"Every website deserves a great search engine - but building a search engine can be a lot of work, and hosting it can quickly get expensive. I’m going to build a search engine for 24 ways that’s fast enough to support autocomplete (a.k.a. typeahead) search queries and can be hosted for free. I’ll be using wget, Python, SQLite, Jupyter, sqlite-utils and my open source Datasette tool to build the API backend, and a few dozen lines of modern vanilla JavaScript to build the interface. Try it out here, then read on to see how I built it. First step: crawling the data The first step in building a search engine is to grab a copy of the data that you plan to make searchable. There are plenty of potential ways to do this: you might be able to pull it directly from a database, or extract it using an API. If you don’t have access to the raw data, you can imitate Google and write a crawler to extract the data that you need. I’m going to do exactly that against 24 ways: I’ll build a simple crawler using wget, a command-line tool that features a powerful “recursive” mode that’s ideal for scraping websites. We’ll start at the https://24ways.org/archives/ page, which links to an archived index for every year that 24 ways has been running. Then we’ll tell wget to recursively crawl the website, using the --recursive flag. We don’t want to fetch every single page on the site - we’re only interested in the actual articles. Luckily, 24 ways has nicely designed URLs, so we can tell wget that we only care about pages that start with one of the years it has been running, using the -I argument like this: -I /2005,/2006,/2007,/2008,/2009,/2010,/2011,/2012,/2013,/2014,/2015,/2016,/2017 We want to be polite, so let’s wait for 2 seconds between each request rather than hammering the site as fast as we can: --wait 2 The first time I ran this, I accidentally downloaded the comments pages as well. We don’t want those, so let’s exclude them from the crawl using -X ""/*/*/comments"". Finally, it’s useful to be able to run the command multiple times without downloading pages that we have already fetched. We can use the --no-clobber option for this. Tie all of those options together and we get this command: wget --recursive --wait 2 --no-clobber -I /2005,/2006,/2007,/2008,/2009,/2010,/2011,/2012,/2013,/2014,/2015,/2016,/2017 -X ""/*/*/comments"" https://24ways.org/archives/ If you leave this running for a few minutes, you’ll end up with a folder structure something like this: $ find 24ways.org 24ways.org 24ways.org/2013 24ways.org/2013/why-bother-with-accessibility 24ways.org/2013/why-bother-with-accessibility/index.html 24ways.org/2013/levelling-up 24ways.org/2013/levelling-up/index.html 24ways.org/2013/project-hubs 24ways.org/2013/project-hubs/index.html 24ways.org/2013/credits-and-recognition 24ways.org/2013/credits-and-recognition/index.html ... As a quick sanity check, let’s count the number of HTML pages we have retrieved: $ find 24ways.org | grep index.html | wc -l 328 There’s one last step! We got everything up to 2017, but we need to fetch the articles for 2018 (so far) as well. They aren’t linked in the /archives/ yet so we need to point our crawler at the site’s front page instead: wget --recursive --wait 2 --no-clobber -I /2018 -X ""/*/*/comments"" https://24ways.org/ Thanks to --no-clobber, this is safe to run every day in December to pick up any new content. We now have a folder on our computer containing an HTML file for every article that has ever been published on the site! Let’s use them to build ourselves a search index. Building a search index using SQLite There are many tools out there that can be used to build a search engine. You can use an open-source search server like Elasticsearch or Solr, a hosted option like Algolia or Amazon CloudSearch or you can tap into the built-in search features of relational databases like MySQL or PostgreSQL. I’m going to use something that’s less commonly used for web applications but makes for a powerful and extremely inexpensive alternative: SQLite. SQLite is the world’s most widely deployed database, even though many people have never even heard of it. That’s because it’s designed to be used as an embedded database: it’s commonly used by native mobile applications and even runs as part of the default set of apps on the Apple Watch! SQLite has one major limitation: unlike databases like MySQL and PostgreSQL, it isn’t really designed to handle large numbers of concurrent writes. For this reason, most people avoid it for building web applications. This doesn’t matter nearly so much if you are building a search engine for infrequently updated content - say one for a site that only publishes new content on 24 days every year. It turns out SQLite has very powerful full-text search functionality built into the core database - the FTS5 extension. I’ve been doing a lot of work with SQLite recently, and as part of that, I’ve been building a Python utility library to make building new SQLite databases as easy as possible, called sqlite-utils. It’s designed to be used within a Jupyter notebook - an enormously productive way of interacting with Python code that’s similar to the Observable notebooks Natalie described on 24 ways yesterday. If you haven’t used Jupyter before, here’s the fastest way to get up and running with it - assuming you have Python 3 installed on your machine. We can use a Python virtual environment to ensure the software we are installing doesn’t clash with any other installed packages: $ python3 -m venv ./jupyter-venv $ ./jupyter-venv/bin/pip install jupyter # ... lots of installer output # Now lets install some extra packages we will need later $ ./jupyter-venv/bin/pip install beautifulsoup4 sqlite-utils html5lib # And start the notebook web application $ ./jupyter-venv/bin/jupyter-notebook # This will open your browser to Jupyter at http://localhost:8888/ You should now be in the Jupyter web application. Click New -> Python 3 to start a new notebook. A neat thing about Jupyter notebooks is that if you publish them to GitHub (either in a regular repository or as a Gist), it will render them as HTML. This makes them a very powerful way to share annotated code. I’ve published the notebook I used to build the search index on my GitHub account. ​ Here’s the Python code I used to scrape the relevant data from the downloaded HTML files. Check out the notebook for a line-by-line explanation of what’s going on. from pathlib import Path from bs4 import BeautifulSoup as Soup base = Path(""/Users/simonw/Dropbox/Development/24ways-search"") articles = list(base.glob(""*/*/*/*.html"")) # articles is now a list of paths that look like this: # PosixPath('...24ways-search/24ways.org/2013/why-bother-with-accessibility/index.html') docs = [] for path in articles: year = str(path.relative_to(base)).split(""/"")[1] url = 'https://' + str(path.relative_to(base).parent) + '/' soup = Soup(path.open().read(), ""html5lib"") author = soup.select_one("".c-continue"")[""title""].split( ""More information about"" )[1].strip() author_slug = soup.select_one("".c-continue"")[""href""].split( ""/authors/"" )[1].split(""/"")[0] published = soup.select_one("".c-meta time"")[""datetime""] contents = soup.select_one("".e-content"").text.strip() title = soup.find(""title"").text.split("" ◆"")[0] try: topic = soup.select_one( '.c-meta a[href^=""/topics/""]' )[""href""].split(""/topics/"")[1].split(""/"")[0] except TypeError: topic = None docs.append({ ""title"": title, ""contents"": contents, ""year"": year, ""author"": author, ""author_slug"": author_slug, ""published"": published, ""url"": url, ""topic"": topic, }) After running this code, I have a list of Python dictionaries representing each of the documents that I want to add to the index. The list looks something like this: [ { ""title"": ""Why Bother with Accessibility?"", ""contents"": ""Web accessibility (known in other fields as inclus..."", ""year"": ""2013"", ""author"": ""Laura Kalbag"", ""author_slug"": ""laurakalbag"", ""published"": ""2013-12-10T00:00:00+00:00"", ""url"": ""https://24ways.org/2013/why-bother-with-accessibility/"", ""topic"": ""design"" }, { ""title"": ""Levelling Up"", ""contents"": ""Hello, 24 ways. Iu2019m Ashley and I sell property ins..."", ""year"": ""2013"", ""author"": ""Ashley Baxter"", ""author_slug"": ""ashleybaxter"", ""published"": ""2013-12-06T00:00:00+00:00"", ""url"": ""https://24ways.org/2013/levelling-up/"", ""topic"": ""business"" }, ... My sqlite-utils library has the ability to take a list of objects like this and automatically create a SQLite database table with the right schema to store the data. Here’s how to do that using this list of dictionaries. import sqlite_utils db = sqlite_utils.Database(""/tmp/24ways.db"") db[""articles""].insert_all(docs) That’s all there is to it! The library will create a new database and add a table to it called articles with the necessary columns, then insert all of the documents into that table. (I put the database in /tmp/ for the moment - you can move it to a more sensible location later on.) You can inspect the table using the sqlite3 command-line utility (which comes with OS X) like this: $ sqlite3 /tmp/24ways.db sqlite> .headers on sqlite> .mode column sqlite> select title, author, year from articles; title author year ------------------------------ ------------ ---------- Why Bother with Accessibility? Laura Kalbag 2013 Levelling Up Ashley Baxte 2013 Project Hubs: A Home Base for Brad Frost 2013 Credits and Recognition Geri Coady 2013 Managing a Mind Christopher 2013 Run Ragged Mark Boulton 2013 Get Started With GitHub Pages Anna Debenha 2013 Coding Towards Accessibility Charlie Perr 2013 ... There’s one last step to take in our notebook. We know we want to use SQLite’s full-text search feature, and sqlite-utils has a simple convenience method for enabling it for a specified set of columns in a table. We want to be able to search by the title, author and contents fields, so we call the enable_fts() method like this: db[""articles""].enable_fts([""title"", ""author"", ""contents""]) Introducing Datasette Datasette is the open-source tool I’ve been building that makes it easy to both explore SQLite databases and publish them to the internet. We’ve been exploring our new SQLite database using the sqlite3 command-line tool. Wouldn’t it be nice if we could use a more human-friendly interface for that? If you don’t want to install Datasette right now, you can visit https://search-24ways.herokuapp.com/ to try it out against the 24 ways search index data. I’ll show you how to deploy Datasette to Heroku like this later in the article. If you want to install Datasette locally, you can reuse the virtual environment we created to play with Jupyter: ./jupyter-venv/bin/pip install datasette This will install Datasette in the ./jupyter-venv/bin/ folder. You can also install it system-wide using regular pip install datasette. Now you can run Datasette against the 24ways.db file we created earlier like so: ./jupyter-venv/bin/datasette /tmp/24ways.db This will start a local webserver running. Visit http://localhost:8001/ to start interacting with the Datasette web application. If you want to try out Datasette without creating your own 24ways.db file you can download the one I created directly from https://search-24ways.herokuapp.com/24ways-ae60295.db Publishing the database to the internet One of the goals of the Datasette project is to make deploying data-backed APIs to the internet as easy as possible. Datasette has a built-in command for this, datasette publish. If you have an account with Heroku or Zeit Now, you can deploy a database to the internet with a single command. Here’s how I deployed https://search-24ways.herokuapp.com/ (running on Heroku’s free tier) using datasette publish: $ ./jupyter-venv/bin/datasette publish heroku /tmp/24ways.db --name search-24ways -----> Python app detected -----> Installing requirements with pip -----> Running post-compile hook -----> Discovering process types Procfile declares types -> web -----> Compressing... Done: 47.1M -----> Launching... Released v8 https://search-24ways.herokuapp.com/ deployed to Heroku If you try this out, you’ll need to pick a different --name, since I’ve already taken search-24ways. You can run this command as many times as you like to deploy updated versions of the underlying database. Searching and faceting Datasette can detect tables with SQLite full-text search configured, and will add a search box directly to the page. Take a look at http://search-24ways.herokuapp.com/24ways-b607e21/articles to see this in action. ​ SQLite search supports wildcards, so if you want autocomplete-style search where you don’t need to enter full words to start getting results you can add a * to the end of your search term. Here’s a search for access* which returns articles on accessibility: http://search-24ways.herokuapp.com/24ways-ae60295/articles?_search=acces%2A A neat feature of Datasette is the ability to calculate facets against your data. Here’s a page showing search results for svg with facet counts calculated against both the year and the topic columns: http://search-24ways.herokuapp.com/24ways-ae60295/articles?_search=svg&_facet=year&_facet=topic Every page visible via Datasette has a corresponding JSON API, which can be accessed using the JSON link on the page - or by adding a .json extension to the URL: http://search-24ways.herokuapp.com/24ways-ae60295/articles.json?_search=acces%2A Better search using custom SQL The search results we get back from ../articles?_search=svg are OK, but the order they are returned in is not ideal - they’re actually being returned in the order they were inserted into the database! You can see why this is happening by clicking the View and edit SQL link on that search results page. This exposes the underlying SQL query, which looks like this: select rowid, * from articles where rowid in ( select rowid from articles_fts where articles_fts match :search ) order by rowid limit 101 We can do better than this by constructing a custom SQL query. Here’s the query we will use instead: select snippet(articles_fts, -1, 'b4de2a49c8', '8c94a2ed4b', '...', 100) as snippet, articles_fts.rank, articles.title, articles.url, articles.author, articles.year from articles join articles_fts on articles.rowid = articles_fts.rowid where articles_fts match :search || ""*"" order by rank limit 10; You can try this query out directly - since Datasette opens the underling SQLite database in read-only mode and enforces a one second time limit on queries, it’s safe to allow users to provide arbitrary SQL select queries for Datasette to execute. There’s a lot going on here! Let’s break the SQL down line-by-line: select snippet(articles_fts, -1, 'b4de2a49c8', '8c94a2ed4b', '...', 100) as snippet, We’re using snippet(), a built-in SQLite function, to generate a snippet highlighting the words that matched the query. We use two unique strings that I made up to mark the beginning and end of each match - you’ll see why in the JavaScript later on. articles_fts.rank, articles.title, articles.url, articles.author, articles.year These are the other fields we need back - most of them are from the articles table but we retrieve the rank (representing the strength of the search match) from the magical articles_fts table. from articles join articles_fts on articles.rowid = articles_fts.rowid articles is the table containing our data. articles_fts is a magic SQLite virtual table which implements full-text search - we need to join against it to be able to query it. where articles_fts match :search || ""*"" order by rank limit 10; :search || ""*"" takes the ?search= argument from the page querystring and adds a * to the end of it, giving us the wildcard search that we want for autocomplete. We then match that against the articles_fts table using the match operator. Finally, we order by rank so that the best matching results are returned at the top - and limit to the first 10 results. How do we turn this into an API? As before, the secret is to add the .json extension. Datasette actually supports multiple shapes of JSON - we’re going to use ?_shape=array to get back a plain array of objects: JSON API call to search for articles matching SVG The HTML version of that page shows the time taken to execute the SQL in the footer. Hitting refresh a few times, I get response times between 2 and 5ms - easily fast enough to power a responsive autocomplete feature. A simple JavaScript autocomplete search interface I considered building this using React or Svelte or another of the myriad of JavaScript framework options available today, but then I remembered that vanilla JavaScript in 2018 is a very productive environment all on its own. We need a few small utility functions: first, a classic debounce function adapted from this one by David Walsh: function debounce(func, wait, immediate) { let timeout; return function() { let context = this, args = arguments; let later = () => { timeout = null; if (!immediate) func.apply(context, args); }; let callNow = immediate && !timeout; clearTimeout(timeout); timeout = setTimeout(later, wait); if (callNow) func.apply(context, args); }; }; We’ll use this to only send fetch() requests a maximum of once every 100ms while the user is typing. Since we’re rendering data that might include HTML tags (24 ways is a site about web development after all), we need an HTML escaping function. I’m amazed that browsers still don’t bundle a default one of these: const htmlEscape = (s) => s.replace( />/g, '>' ).replace( /Autocomplete search

And now the autocomplete implementation itself, as a glorious, messy stream-of-consciousness of JavaScript: // Embed the SQL query in a multi-line backtick string: const sql = `select snippet(articles_fts, -1, 'b4de2a49c8', '8c94a2ed4b', '...', 100) as snippet, articles_fts.rank, articles.title, articles.url, articles.author, articles.year from articles join articles_fts on articles.rowid = articles_fts.rowid where articles_fts match :search || ""*"" order by rank limit 10`; // Grab a reference to the const searchbox = document.getElementById(""searchbox""); // Used to avoid race-conditions: let requestInFlight = null; searchbox.onkeyup = debounce(() => { const q = searchbox.value; // Construct the API URL, using encodeURIComponent() for the parameters const url = ( ""https://search-24ways.herokuapp.com/24ways-866073b.json?sql="" + encodeURIComponent(sql) + `&search=${encodeURIComponent(q)}&_shape=array` ); // Unique object used just for race-condition comparison let currentRequest = {}; requestInFlight = currentRequest; fetch(url).then(r => r.json()).then(d => { if (requestInFlight !== currentRequest) { // Avoid race conditions where a slow request returns // after a faster one. return; } let results = d.map(r => `

${htmlEscape(r.title)}

${htmlEscape(r.author)} - ${r.year}

${highlight(r.snippet)}

`).join(""""); document.getElementById(""results"").innerHTML = results; }); }, 100); // debounce every 100ms There’s just one more utility function, used to help construct the HTML results: const highlight = (s) => htmlEscape(s).replace( /b4de2a49c8/g, '' ).replace( /8c94a2ed4b/g, '' ); This is what those unique strings passed to the snippet() function were for. Avoiding race conditions in autocomplete One trick in this code that you may not have seen before is the way race-conditions are handled. Any time you build an autocomplete feature, you have to consider the following case: User types acces Browser sends request A - querying documents matching acces* User continues to type accessibility Browser sends request B - querying documents matching accessibility* Request B returns. It was fast, because there are fewer documents matching the full term The results interface updates with the documents from request B, matching accessibility* Request A returns results (this was the slower of the two requests) The results interface updates with the documents from request A - results matching access* This is a terrible user experience: the user saw their desired results for a brief second, and then had them snatched away and replaced with those results from earlier on. Thankfully there’s an easy way to avoid this. I set up a variable in the outer scope called requestInFlight, initially set to null. Any time I start a new fetch() request, I create a new currentRequest = {} object and assign it to the outer requestInFlight as well. When the fetch() completes, I use requestInFlight !== currentRequest to sanity check that the currentRequest object is strictly identical to the one that was in flight. If a new request has been triggered since we started the current request we can detect that and avoid updating the results. It’s not a lot of code, really And that’s the whole thing! The code is pretty ugly, but when the entire implementation clocks in at fewer than 70 lines of JavaScript, I honestly don’t think it matters. You’re welcome to refactor it as much you like. How good is this search implementation? I’ve been building search engines for a long time using a wide variety of technologies and I’m happy to report that using SQLite in this way is genuinely a really solid option. It scales happily up to hundreds of MBs (or even GBs) of data, and the fact that it’s based on SQL makes it easy and flexible to work with. A surprisingly large number of desktop and mobile applications you use every day implement their search feature on top of SQLite. More importantly though, I hope that this demonstrates that using Datasette for an API means you can build relatively sophisticated API-backed applications with very little backend programming effort. If you’re working with a small-to-medium amount of data that changes infrequently, you may not need a more expensive database. Datasette-powered applications easily fit within the free tier of both Heroku and Zeit Now. For more of my writing on Datasette, check out the datasette tag on my blog. And if you do build something fun with it, please let me know on Twitter.",2018,Simon Willison,simonwillison,2018-12-19T00:00:00+00:00,https://24ways.org/2018/fast-autocomplete-search-for-your-website/,code 253,Clip Paths Know No Bounds,"CSS Shapes are getting a lot of attention as browser support has increased for properties like shape-outside and clip-path. There are a few ways that we can use CSS Shapes, in particular with the clip-path property, that are not necessarily evident at first glance. The basics of a clip path Before we dig into specific techniques to expand on clip paths, we should first take a look at a basic shape and clip-path. Clip paths can apply a CSS Shape such as a circle(), ellipse(), inset(), or the flexible polygon() to any element. Everywhere in the element that is not within the bounds of our shape will be visually removed. Using the polygon shape function, for example, we can create triangles, stars, or other straight-edged shapes as on Bennett Feely’s Clippy. While fixed units like pixels can be used when defining vertices/points (where the sides meet), percentages will give more flexibility to adapt to the element’s dimensions. See the Pen Clip Path Box by Dan Wilson (@danwilson) on CodePen. So for an octagon, we can set eight x, y pairs of percentages to define those points. In this case we start 30% into the width of the box for the first x and at the top of the box for the y and go clockwise. The visible area becomes the interior of the shape made by connecting these points with straight lines. clip-path: polygon( 30% 0%, 70% 0%, 100% 30%, 100% 70%, 70% 100%, 30% 100%, 0% 70%, 0% 30% ); A shape with less vertices than the eye can see It’s reasonable to look at the polygon() function and assume that we need to have one pair of x, y coordinates for every point in our shape. However, we gain some flexibility by thinking outside the box — or more specifically when we think outside the range of 0% - 100%. Our element’s box model will be the ultimate boundary for a clip-path, but we can still define points that exist beyond that natural box for an element. See the Pen CSS Shapes Know No Bounds by Dan Wilson (@danwilson) on CodePen. By going beyond the 0% - 100% range we can turn a polygon with three points into a quadrilateral, a pentagon, or a hexagon. In this example the shapes used are all similar triangles defining three points, but due to exceeding the bounds for our element box we visually see one triangle and two pentagons. Our earlier octagon can similarly be made with only four points. See the Pen Octagon with four points by Dan Wilson (@danwilson) on CodePen. Multiple shapes, one clip path We can lean on this power of going beyond the bounds of our element to also create more than one visual shape with a single polygon(). See the Pen Multiple shapes from one clip-path by Dan Wilson (@danwilson) on CodePen. Depending on how we lay it out we can make each shape directly, but since we know we can move around in the space beyond the element’s box, we can draw extra lines to help us get where we need to go next as needed. It can also help us in slicing an element. Combined with CSS Variables, we can work with overlapping elements and clip each one into alternating strips. This example is two elements, each divided into a few rectangles. See the Pen 24w: Sliced Icon by Dan Wilson (@danwilson) on CodePen. Different shapes with fill rules A polygon() is not just a collection of points. There is one more key piece to its puzzle according to the specification — the Fill Rule. The default value we have been using so far is nonzero, and the second option is evenodd. These two values help determine what is considered inside and outside the shape. See the Pen A Star Multiways by Dan Wilson (@danwilson) on CodePen. As lines intersect we can get into situations where pieces seemingly on the inside can be considered outside the shape boundary. When using the evenodd fill rule, we can determine if a given point is inside or outside the boundary by drawing a ray from the point in any direction. If the ray crosses an even number of the clip path’s lines, the point is considered outside, and if it crosses an odd number the point is inside. Order of operations It is important to note that there are many CSS properties that affect the final composited appearance of an element via CSS Filters, Blend Modes, and more. These compositing effects are applied in the order: CSS Filters (e.g. filter: blur(2px)) Clipping (e.g. what this article is about) Masking (Clipping’s cousin) Blend Modes (e.g. mix-blend-mode: multiply) Opacity This means if we want to have a star shape and blur it, the blur will happen before the clip. And since blurs are most noticeable around the edge of an element box, the effect might be completely lost since we have clipped away the element’s box edges. See the Pen Order of Filter + Clip by Dan Wilson (@danwilson) on CodePen. If we want the edges of the star to be blurred, we do have the option to wrap our clipped element in a blurred parent element. The inner element will be rendered first (with its star clip) and then the parent will blur its contents normally. Revealing content with animation CSS Shapes can be transitioned and animated, allowing us to animate the visual area of our element without affecting the content within. For example, we can start with visually hidden content (fully clipped) and grow the clip path to reveal the content within. The important caveat for polygon() is that the number of points need to be the same for each keyframe, as well as the fill rule. Otherwise the browser will not have enough information to interpolate the intermediate values. See the Pen Clip Path Shape Reveal by Dan Wilson (@danwilson) on CodePen. Don’t keep CSS Shapes in a box Clip paths give us some interesting new possibilities, especially when we think of them as more than just basic shapes. We may be heavily modifying the visual representation of our elements with clip-path, but the underlying content remains unchanged and accessible which makes this property fairly powerful.",2018,Dan Wilson,danwilson,2018-12-20T00:00:00+00:00,https://24ways.org/2018/clip-paths-know-no-bounds/,code 255,Inclusive Considerations When Restyling Form Controls,"I would like to begin by saying 2018 was the year that we, as developers, visual designers, browser implementers, and inclusive design and experience specialists rallied together and achieved a long-sought goal: We now have the ability to fully style form controls, across all modern browsers, while retaining their ease of declaration, native functionality and accessibility. I would like to begin by saying all these things. However, they’re not true. I think we spent the year debating about what file extension CSS should be written in, or something. Or was that last year? Maybe I’m thinking of next year. Returning to reality, styling form controls is more tricky and time consuming these days rather than flat out “hard”. In fact, depending on the length of the styling-leash a particular browser provides, there are controls you can style quite a bit. As for browsers with shorter leashes, there are other options to force their controls closer to the visual design you’re tasked to match. However, when striving for custom styled controls, one must be careful not to forget about the inherent functionality and accessibility that many provide. People expect and deserve the products and services they use and pay for to work for them. If these services are visually pleasing, but only function for those who fit the handful of personas they’ve been designed for, then we’ve potentially deprived many people the experiences they deserve. Quick level setting Getting down to brass tacks, when creating custom styled form controls that should retain their expected semantics and functionality, we have to consider the following: Many form elements can be styled directly through standard and browser specific selectors, as well as through some clever styling of markup patterns. We should leverage these native options before reinventing any wheels. It is important to preserve the underlying semantics of interactive controls. We must not unintentionally exclude people who use assistive technologies (ATs) that rely on these semantics. Make sure you test what you create. There is a lot of underlying complexity to form controls which may not be immediately apparent if they’re judged solely by their visual presentation in a single browser, or with limited AT testing. Visually resetting and restyling form controls Over the course of 2018, I worked on a project where I tested and reported on the accessibility impact of styling various form controls. In conducting my research, I reviewed many of the form controls available in HTML, testing to see how malleable they were to direct styling from standardized CSS selectors. As I expected, controls such as the various text fields could be restyled rather easily. However, other controls like radio buttons and checkboxes, or sub-elements of special text fields like date, search, and number spinners were resistant to standard-based styling. These particular controls and their sub-elements required specific pseudo-elements to reset and allow for restyling of some of their default presentation. See the Pen form control styling comparisons by Scott (@scottohara) on CodePen. https://codepen.io/scottohara/pen/gZOrZm/ Over the years, the ability to directly style form controls has been something many people have clamored for. However, one should realize the benefits of being able to restyle some of these controls may involve more effort than originally anticipated. If you want to restyle a control from the ground up, then you must also recreate any :active, :focus, and :hover states for the control—all those things that were previously taken care of by browsers. Not only that, but anything you restyle should also work with Windows High Contrast mode, styling for dark mode, and other OS-level settings that browser respect without you even realizing. You ever try playing with the accessibility settings of your display on macOS, or similar Windows setting? It is also worth mentioning that any browser prefixed pseudo-elements are not standardized CSS selectors. As MDN mentions at the top of their pages documenting these pseudo-elements: Non-standard This feature is non-standard and is not on a standards track. Do not use it on production sites facing the Web: it will not work for every user. There may also be large incompatibilities between implementations and the behavior may change in the future. While this may be a deterrent for some, it’s my opinion the risks are often only skin-deep. By which I mean if a non-standard selector does change, the control may look a bit quirky, but likely won’t cease to function. A bug report which requires a CSS selector change can be an easy JIRA ticket to close, after all. Can’t make it? Fake it. Internet Explorer 11 (IE11) is still neck-and-neck with other browsers in vying for the number 2 spot in desktop browser share. Due to IE not recognizing vendor-prefixed appearance properties, some essential controls like checkboxes won’t render as intended. Additionally, some controls like select boxes, file uploads, and sub-elements of date fields (calendar popups) cannot be modified by just relying on styling their HTML selectors alone. This means that unless your company designs and develops with a progressive enhancement, or graceful degradation mindset, you’ll need to take a different approach in styling. Getting clever with markup and CSS The following CodePen demonstrates how we can create a custom checkbox markup pattern. By mindfully utilizing CSS sibling selectors and positioning of the native control, we can create custom visual styling while also retaining the functionality and accessibility expectations of a native checkbox. See the Pen Accessible Styled Native Checkbox by Scott (@scottohara) on CodePen. https://codepen.io/scottohara/pen/RqEayN/ Customizing checkboxes by visually hiding the input and styling well-placed markup with sibling selectors may seem old hat to some. However, many variations of these patterns do not take into account how their method of visually hiding the checkboxes can create discovery issues for certain screen reader navigation methods. For instance, if someone is using a mobile device and exploring by touch, how will they be able to drag their finger over an input that has been reduced to a single pixel, or positioned off screen? As we move away from the simplicity of declaring a single HTML element and using clever CSS and markup patterns to create restyled form controls, we increase the need for additional testing to ensure no expected behaviors are lost. In other words, what should work in theory may not work in practice when you introduce the various different ways people may engage with a form control. It’s worth remembering: what might be typical interactions for ourselves may be problematic if not impossible for others. Limitations to cleverness Creative coding will allow us to apply more consistent custom styles to some of the more problematic form controls. There will be a varied amount of custom markup, CSS, and sometimes JavaScript that will be needed to preserve the control’s inherent usability and accessibility for each control we take this approach to. However, this method of restyling still doesn’t solve for the lack of feature parity across different browsers. Nor is it a means to account for controls which don’t have a native HTML element equivalent, such as a switch or multi-thumb range slider? Maybe there’s a control that calls for a visual design or proposed user experience that would require too much fighting with a native control’s behavior to be worth the level of effort to implement. Here’s where we need to take another approach. Using ARIA when appropriate Sometimes we have no other option than to roll up our sleeves and start building custom form controls from scratch. Fair warning though: just because we’re not leveraging a native HTML control as our foundation, it doesn’t mean we have carte blanche to throw semantics out the window. Enter Accessible Rich Internet Applications (ARIA). ARIA is a set of attributes that can modify existing elements, or extend HTML to include roles, properties and states that aren’t native to the language. While divs and spans have no meaningful semantic information for us to leverage, with help from the ARIA specification and ARIA Authoring Practices we can incorporate these elements to help create the UI that we need while still following the first rule of Using ARIA: If you can use a native HTML element or attribute with the semantics and behavior you require already built in, instead of re-purposing an element and adding an ARIA role, state or property to make it accessible, then do so. By using these documents as guidelines, and testing our custom controls with people of various abilities, we can do our best to make sure a custom control performs as expected for as many people as possible. Exceptions to the rule One example of a control that allows for an exception to the first rule of Using ARIA would be a switch control. Switches and checkboxes are similar components, in that they have both on/checked and off/unchecked states. However, checkboxes are often expected within the context of forms, or used to filter search queries on e-commerce sites. Switches are typically used to instantly enable or deactivate a particular setting at a component or app-based level, as this is their behavior in the native mobile apps in which they were popularized. While a switch control could be created by visually restyling a checkbox, this does not automatically mean that the underlying semantics and functionality will match the visual representation of the control. For example, the following CodePen restyles checkboxes to look like a switch control, but the semantics of the checkboxes remain which communicate a different way of interacting with the control than what you might expect from a native switch control. See the Pen Switch Boxes - custom styled checkboxes posing as switches by Scott (@scottohara) on CodePen. https://codepen.io/scottohara/pen/XyvoeE/ By adding a role=""switch"" to these checkboxes, we can repurpose the inherent checked/unchecked states of the native control, it’s inherent ability to be focused by Tab key, and Space key to toggle state. But while this is a valid approach to take in building a switch, how does this actually match up to reality? Does it pass the test(s)? Whether deconstructing form controls to fully restyle them, or leveraging them and other HTML elements as a base to expand on, or create, a non-native form control, building it is just the start. We must test that what we’ve restyled or rebuilt works the way people expect it to, if not better. What we must do here is run a gamut of comparative tests to document the functionality and usability of native form controls. For example: Is the control implemented in all supported browsers? If not: where are the gaps? Will it be necessary to implement a custom solution for the situations that degrade to a standard text field? If so: is each browser’s implementation a good user experience? Is there room for improvement that can be tested against the native baseline? Test with multiple input devices. Where the control is implemented, what is the quality of the user experience when using different input devices, such as mouse, touchscreen, keyboard, speech recognition or switch device, to name a few. You’ll find some HTML5 controls (like date pickers and number spinners) have additional UI elements that may not be announced to AT, or even allow keyboard accessibility. Often these controls can be adjusted by other means, such as text entry, or using arrow keys to increase or decrease values. If restyling or recreating a custom version of a control like these, it may make sense to maintain these native experiences as well. How well does the control take to custom styles? If a control can be styled enough to not need to be rebuilt from scratch, that’s great! But make sure that there are no adverse affects on the accessibility of it. For instance, range sliders can be restyled and maintain their functionality and accessibility. However, elements like progress bars can be negatively affected by direct styling. Always test with different browser and AT pairings to ensure nothing is lost when controls are restyled. Do specifications match reality? If recreating controls to get around native limitations, such as the inability to style the options of a select element, or requiring a Switch control which is not native to HTML, do your solutions match user expectations? For instance, selects have unique picker interfaces on touch devices. And switches have varied levels of support for different browser and screen reader pairings. Test with real people, and check your analytics. If these experiences don’t match people’s expectations, then maybe another solution is in order? Wrapping up While styling form controls is definitely easier than it’s ever been, that doesn’t mean that it’s at all simple, nor will it likely ever be. The level of difficulty you’re going to face is going to depend entirely on what it is you’re hoping to style, add-on to, or recreate. And even if you build your custom control exactly to specification, you’ll still be reliant on browsers and assistive technologies being able to fully understand the component they’ve been presented. Forms and their controls are an incredibly important part of what we need the Internet for. Paying bills, scheduling appointments, ordering groceries, renewing your license or even ordering gifts for the holidays. These are all important tasks that people should be able to complete with as little effort as possible. Especially since for some, completing these tasks online might be their only option. 2018 didn’t end up being the year we got full customization of form controls sorted out. But that’s OK. If we can continue to mindfully work with what we have, and instead challenge ourselves to follow inclusive design principles, well thought out Form Design Patterns, and solve problems with an accessibility first approach, we may come to realize that we can get along just fine without fully branded drop downs. And hey. There’s always next year, right?",2018,Scott O'Hara,scottohara,2018-12-13T00:00:00+00:00,https://24ways.org/2018/inclusive-considerations-when-restyling-form-controls/,code 256,Develop Your Naturalist Superpowers with Observable Notebooks and iNaturalist,"We’re going to level up your knowledge of what animals you might see in an area at a particular time of year - a skill every naturalist* strives for - using technology! Using iNaturalist and Observable Notebooks we’re going to prototype seasonality graphs for particular species in an area, and automatically create a guide to what animals you might see in each month. *(a Naturalist is someone who likes learning about nature, not someone who’s a fan of being naked, that’s a ‘Naturist’… different thing!) Looking for critters in rocky intertidal habitats One of my favourite things to do is going rockpooling, or as we call it over here in California, ‘tidepooling’. Amounting to the same thing, it’s going to a beach that has rocks where the tide covers then uncovers little pools of water at different times of the day. All sorts of fun creatures and life can be found in this ‘rocky intertidal habitat’ A particularly exciting creature that lives here is the Nudibranch, a type of super colourful ‘sea slug’. There are over 3000 species of Nudibranch worldwide. (The word “nudibranch” comes from the Latin nudus, naked, and the Greek βρανχια / brankhia, gills.) ​ They are however quite tricky to find! Even though they are often brightly coloured and interestingly shaped, some of them are very small, and in our part of the world in the Bay Area in California their appearance in our rockpools is seasonal. We see them more often in Summer months, despite the not-as-low tides as in our Winter and Spring seasons. My favourite place to go tidepooling here is Pillar Point in Half Moon bay (at other times of the year more famously known for the surf competition ‘Mavericks’). The rockpools there are rich in species diversity, of varied types and water-coverage habitat zones as well as being relatively accessible. ​ I was rockpooling at Pillar Point recently with my parents and we talked to a lady who remarked that she hadn’t seen any Nudibranchs on her visit this time. I realised that having an idea of what species to find where, and at what time of year is one of the many superpower goals of every budding Naturalist. Using technology and the croudsourced species observations of the iNaturalist community we can shortcut our way to this superpower! Finding nearby animals with iNaturalist We’re going to be getting our information about what animals you can see in Pillar Point using iNaturalist. iNaturalist is a really fun platform that helps connect people to nature and report their findings of life in the outdoors. It is also a community of nature-loving people who help each other identify and confirm those observations. iNaturalist is a project run as a joint initiative by the California Academy of Sciences and the National Geographic Society. I’ve been using iNaturalist for over two years to record and identify plants and animals that I’ve found in the outdoors. I use their iPhone app to upload my pictures, which then uses machine learning algorithms to make an initial guess at what it is I’ve seen. The community is really active, and I often find someone else has verified or updated my species guess pretty soon after posting. This process is great because once an observation has been identified by at least two people it becomes ‘verified’ and is considered research grade. Research grade observations get exported and used by scientists, as well as being indexed by the Global Biodiversity Information Facility, GBIF. ​ iNaturalist has a great API and API explorer, which makes interacting and prototyping using iNaturalist data really fun. For example, if you go to the API explorer and expand the Observations : Search and fetch section and then the GET /observations API, you get a selection of input boxes that allow you to play with options that you can then pass to the API when you click the ‘Try it out’ button. ​ You’ll then get a URL that looks a bit like https://api.inaturalist.org/v1/observations?captive=false &geo=true&verifiable=true&taxon_id=47113&lat=37.495461&lng=-122.499584 &radius=5&order=desc&order_by=created_at which you can call and interrrogate using a programming language of your choice. If you would like to see an all-JavaScript application that uses the iNaturalist API, take a look at OwlsNearMe.com which Simon and I built one weekend earlier this year. It gets your location and shows you all iNaturalist observations of owls near you and lists which species you are likely to see (not adjusted for season). Rapid development using Observable Notebooks We’re going to be using Observable Notebooks to prototype our examples, pulling data down from iNaturalist. I really like using visual notebooks like Observable, they are great for learning and building things quickly. You may be familiar with Jupyter notebooks for Python which is similar but takes a bit of setup to get going - I often use these for prototyping too. Observable is amazing for querying and visualising data with JavaScript and since it is a hosted product it doesn’t require any setup at all. You can follow along and play with this example on my Observable notebook. If you create an account there you can fork my notebook and create your own version of this example. Each ‘notebook’ consists of a page with a column of ‘cells’, similar to what you get in a spreadsheet. A cell can contain Markdown text or JavaScript code and the output of evaluating the cell appears above the code that generated it. There are lots of tutorials out there on Observable Notebooks, I like this code introduction one from Observable (and D3) creator Mike Bostock. Developing your Naturalist superpowers If you have an idea of what plants and critters you might see in a place at the time you visit, you can hone in on what you want to study and train your Naturalist eye to better identify the life around you. For our example, we care about wildlife we can see at Pillar Point, so we need a way of letting the iNaturalist API know which area we are interested in. We could use a latitide, longitude and radius for this, but a rectangular bounding box is a better shape for the reef. We can use this tool to draw the area we want to search within: boundingbox.klokantech.com ​ The tool lets you export the bounding box in several forms using the dropdown at the bottom left under the map givese We are going to use the ‘DublinCore’ format as it’s closest to the format needed by the iNaturalist API. westlimit=-122.50542; southlimit=37.492805; eastlimit=-122.492738; northlimit=37.499811 A quick map primer: The higher the latitude the more north it is The lower the latitude the more south it is Latitude 0 = the equator The higher the longitude the more east it is of Greenwich The lower the longitude the more west it is of Greenwich Longitude 0 = Greenwich In the iNaturalst API we want to use the parameters nelat, nelng, swlat, swlng to create a query that looks inside a bounding box of Pillar Point near Half Moon Bay in California: nelat = highest latitude = north limit = 37.499811 nelng = highest longitude = east limit = -122.492738 swlat = smallest latitude = south limit = 37.492805 swlng = smallest longitude = west limit = 122.50542 As API parameters these look like this: ?nelat=37.499811&nelng=-122.492738&swlat=37.492805&swlng=122.50542 These parameters in this format can be used for most of the iNaturalist API methods. Nudibranch seasonality in Pillar Point We can use the iNaturalist observation_histogram API to get a count of Nudibranch observations per week-of-year across all time and within our Pillar Point bounding box. In addition to the geographic parameters that we just worked out, we are also sending the taxon_id of 47113, which is iNaturalists internal number associated with the Nudibranch taxon. By using this we can get all species which are under the parent ‘Order Nudibranchia’. Another useful piece of naturalist knowledge is understanding the biological classification scheme of Taxanomic Rank - roughly, when a species has a Latin name of two words eg ‘Glaucus Atlanticus’ the first Latin word is the ‘Genus’ like a family name ‘Glaucus’, and the second word identifies that particular species, like a given name ‘Atlanticus’. The two Latin words together indicate a specific species, the term we use colloquially to refer to a type of animal often differs wildly region to region, and sometimes the same common name in two countries can refer to two different species. The common names for the Glaucus Atlanticus (which incidentally is my favourite sea slug) include: sea swallow, blue angel, blue glaucus, blue dragon, blue sea slug and blue ocean slug! Because this gets super confusing, Scientists like using this Latin name format instead. The following piece of code asks the iNaturalist Histogram API to return per-week counts for verified observations of Nudibranchs within our Pillar Point bounding box: pillar_point_counts_per_week = fetch( ""https://api.inaturalist.org/v1/observations/histogram?taxon_id=47113&nelat=37.499811&nelng=-122.492738&swlat=37.492805&swlng=-122.50542&date_field=observed&interval=week_of_year&verifiable=true"" ).then(response => { return response.json(); }) Our next step is to take this data and draw a graph! We’ll be using Vega-Lite for this, which is a fab JavaScript graphing libary that is also easy and fun to use with Observable Notebooks. (Here is a great tutorial on exploring data and drawing graphs with Observable and Vega-Lite) The iNaturalist API returns data that looks like this: { ""total_results"": 53, ""page"": 1, ""per_page"": 53, ""results"": { ""week_of_year"": { ""1"": 136, ""2"": 20, ""3"": 150, ""4"": 65, ""5"": 186, ""6"": 74, ""7"": 47, ""8"": 87, ""9"": 64, ""10"": 56, But for our Vega-Lite graph we need data that looks like this: [{ ""week"": ""01"", ""value"": 136 }, { ""week"": ""02"", ""value"": 20 }, ...] We can convert what we get back from the API to the second format using a loop that iterates over the object keys: objects_to_plot = { let objects = []; Object.keys(pillar_point_counts_per_week.results.week_of_year).map(function(week_index) { objects.push({ week: `Wk ${week_index.toString()}`, observations: pillar_point_counts_per_week.results.week_of_year[week_index] }); }) return objects; } We can then plug this into Vega-Lite to draw us a graph: vegalite({ data: {values: objects_to_plot}, mark: ""bar"", encoding: { x: {field: ""week"", type: ""nominal"", sort: null}, y: {field: ""observations"", type: ""quantitative""} }, width: width * 0.9 }) It’s worth noting that we have a lot of observations of Nudibranchs particularly at Pillar Point due in no small part to the intertidal monitoring research that Alison Young and Rebecca Johnson facilitate for the California Achademy of Sciences. So, what if we want to look for the seasonality of observations of a particular species of adorable sea slug? We want our interface to have a select box with a list of all the species you might find at any time of year. We can do this using the species_counts API to create us an object with the iNaturalist species ID and common & Latin names. pillar_point_nudibranches = { let api_results = await fetch( ""https://api.inaturalist.org/v1/observations/species_counts?taxon_id=47113&nelat=37.499811&nelng=-122.492738&swlat=37.492805&swlng=-122.50542&date_field=observed&verifiable=true"" ).then(r => r.json()) let species_list = api_results.results.map(i => ({ value: i.taxon.id, label: `${i.taxon.preferred_common_name} (${i.taxon.name})` })); return species_list } We can create an interactive select box by importing code from Jeremy Ashkanas’ Observable Notebook: add import {select} from ""@jashkenas/inputs"" to a cell anywhere in our notebook. Observable is magic: like a spreadsheet, the order of the cells doesn’t matter - if one cell is referenced by any other cell then when that cell updates all the other cells refresh themselves. You can also import and reference one notebook from another! viewof select_species = select({ title: ""Which Nudibranch do you want to see seasonality for?"", options: [{value: """", label: ""All the Nudibranchs!""}, ...pillar_point_nudibranches], value: """" }) Then we go back to our old favourite, the histogram API just like before, only this time we are calling it with the value created by our select box ${select_species} as taxon_id instead of the number 47113. pillar_point_counts_per_month_per_species = fetch( `https://api.inaturalist.org/v1/observations/histogram?taxon_id=${select_species}&nelat=37.499811&nelng=-122.492738&swlat=37.492805&swlng=-122.50542&date_field=observed&interval=month_of_year&verifiable=true` ).then(r => r.json()) Now for the fun graph bit! As we did before, we re-format the result of the API into a format compatible with Vega-Lite: objects_to_plot_species_month = { let objects = []; Object.keys(pillar_point_counts_per_month_per_species.results.month_of_year).map(function(month_index) { objects.push({ month: (new Date(2018, (month_index - 1), 1)).toLocaleString(""en"", {month: ""long""}), observations: pillar_point_counts_per_month_per_species.results.month_of_year[month_index] }); }) return objects; } (Note that in the above code we are creating a date object with our specific month in, and using toLocalString() to get the longer English name for the month. Because the JavaScript Date object counts January as 0, we use month_index -1 to get the correct month) And we draw the graph as we did before, only now if you interact with the select box in Observable the graph will dynamically update! vegalite({ data: {values: objects_to_plot_species_month}, mark: ""bar"", encoding: { x: {field: ""month"", type: ""nominal"", sort:null}, y: {field: ""observations"", type: ""quantitative""} }, width: width * 0.9 }) Now we can see when is the best time of year to plan to go tidepooling in Pillar Point if we want to find a specific species of Nudibranch. ​ This tool is great for planning when we to go rockpooling at Pillar Point, but what about if you are going this month and want to pre-train your eye with what to look for in order to impress your friends with your knowledge of Nudibranchs? Well… we can create ourselves a dynamic guide that you can with a list of the species, their photo, name and how many times they have been observed in that month of the year! Our select box this time looks as follows, simpler than before but assigning the month value to the variable selected_month. viewof selected_month = select({ title: ""When do you want to see Nudibranchs?"", options: [ { label: ""Whenever"", value: """" }, { label: ""January"", value: ""1"" }, { label: ""February"", value: ""2"" }, { label: ""March"", value: ""3"" }, { label: ""April"", value: ""4"" }, { label: ""May"", value: ""5"" }, { label: ""June"", value: ""6"" }, { label: ""July"", value: ""7"" }, { label: ""August"", value: ""8"" }, { label: ""September"", value: ""9"" }, { label: ""October"", value: ""10"" }, { label: ""November"", value: ""11"" }, { label: ""December"", value: ""12"" }, ], value: """" }) We then can use the species_counts API to get all the relevant information about which species we can see in month=${selected_month}. We’ll be able to reference this response object and its values later with the variable we just created, eg: all_species_data.results[0].taxon.name. all_species_data = fetch( `https://api.inaturalist.org/v1/observations/species_counts?taxon_id=47113&month=${selected_month}&nelat=37.499811&nelng=-122.492738&swlat=37.492805&swlng=-122.50542&verifiable=true` ).then(r => r.json()) You can render HTML directly in a notebook cell using Observable’s html tagged template literal:

If you go to Pillar Point ${ {"""": """", ""1"":""in January"", ""2"":""in Febrary"", ""3"":""in March"", ""4"":""in April"", ""5"":""in May"", ""6"":""in June"", ""7"":""in July"", ""8"":""in August"", ""9"":""in September"", ""10"":""in October"", ""11"":""in November"", ""12"":""in December"", }[selected_month] } you might see…

${all_species_data.results.map(s => `

${s.taxon.name}

Seen ${s.count} times

`)}
These few lines of HTML are all you need to get this exciting dynamic guide to what Nudibranchs you will see in each month! ​ Play with it yourself in this Observable Notebook. Conclusion I hope by playing with these examples you have an idea of how powerful it can be to prototype using Observable Notebooks and how you can use the incredible crowdsourced community data and APIs from iNaturalist to augment your naturalist skills and impress your friends with your new ‘knowledge of nature’ superpower. Lastly I strongly encourage you to get outside on a low tide to explore your local rocky intertidal habitat, and all the amazing critters that live there. Here is a great introduction video to tidepooling / rockpooling, by Rebecca Johnson and Alison Young from the California Academy of Sciences.",2018,Natalie Downe,nataliedowne,2018-12-18T00:00:00+00:00,https://24ways.org/2018/observable-notebooks-and-inaturalist/,code 257,The (Switch)-Case for State Machines in User Interfaces,"You’re tasked with creating a login form. Email, password, submit button, done. “This will be easy,” you think to yourself. Login form by Selecto You’ve made similar forms many times in the past; it’s essentially muscle memory at this point. You’re working closely with a designer, who gives you a beautiful, detailed mockup of a login form. Sure, you’ll have to translate the pixels to meaningful, responsive CSS values, but that’s the least of your problems. As you’re writing up the HTML structure and CSS layout and styles for this form, you realize that you don’t know what the successful “logged in” page looks like. You remind the designer, who readily gives it to you. But then you start thinking more and more about how the login form is supposed to work. What if login fails? Where do those errors show up? Should we show errors differently if the user forgot to enter their email, or password, or both? Or should the submit button be disabled? Should we validate the email field? When should we show validation errors – as they’re typing their email, or when they move to the password field, or when they click submit? (Note: many, many login forms are guilty of this.) When should the errors disappear? What do we show during the login process? Some loading spinner? What if loading takes too long, or a server error occurs? Many more questions come up, and you (and your designer) are understandably frustrated. The lack of upfront specification opens the door to scope creep, which readily finds itself at home in all the unexplored edge cases. Modeling Behavior Describing all the possible user flows and business logic of an application can become tricky. Ironically, user stories might not tell the whole story – they often leave out potential edge-cases or small yet important bits of information. However, one important (and very old) mathematical model of computation can be used for describing the behavior and all possible states of a user interface: the finite state machine. The general idea, as it applies to user interfaces, is that all of our applications can be described (at some level of abstraction) as being in one, and only one, of a finite number of states at any given time. For example, we can describe our login form above in these states: start - not submitted yet loading - submitted and logging in success - successfully logged in error - login failed Additionally, we can describe an application as accepting a finite number of events – that is, all the possible events that can be “sent” to the application, either from the user or some other external entity: SUBMIT - pressing the submit button RESOLVE - the server responds, indicating that login is successful REJECT - the server responds, indicating that login failed Then, we can combine these states and events to describe the transitions between them. That is, when the application is in one state, an an event occurs, we can specify what the next state should be: From the start state, when the SUBMIT event occurs, the app should be in the loading state. From the loading state, when the RESOLVE event occurs, login succeeded and the app should be in the success state. If login fails from the loading state (i.e., when the REJECT event occurs), the app should be in the error state. From the error state, the user should be able to retry login: when the SUBMIT event occurs here, the app should go to the loading state. Otherwise, if any other event occurs, don’t do anything and stay in the same state. That’s a pretty thorough description, similar to a user story! It’s also a bit more symbolic than a user story (e.g., “when the SUBMIT event occurs” instead of “when the user presses the submit button”), and that’s for a reason. By representing states, events, and transitions symbolically, we can visualize what this state machine looks like: Every state is represented by a box, and every event is connected to a transition arrow that connects two states. This makes it intuitive to follow the flow and understand what the next state should be given the current state and an event. From Visuals to Code Drawing a state machine doesn’t require any special software; in fact, using paper and pencil (in case anything changes!) does the job quite nicely. However, one common problem is handoff: it doesn’t matter how detailed a user story or how well-designed a visualization is, it eventually has to be coded in order for it to become part of a real application. With the state machine model described above, the same visual description can be mapped directly to code. Traditionally, and as the title suggests, this is done using switch/case statements: function loginMachine(state, event) { switch (state) { case 'start': if (event === 'SUBMIT') { return 'loading'; } break; case 'loading': if (event === 'RESOLVE') { return 'success'; } else if (event === 'REJECT') { return 'error'; } break; case 'success': // Accept no further events break; case 'error': if (event === 'SUBMIT') { return 'loading'; } break; default: // This should never occur return undefined; } } console.log(loginMachine('start', 'SUBMIT')); // => 'loading' This is fine (I suppose) but personally, I find it much easier to use objects: const loginMachine = { initial: ""start"", states: { start: { on: { SUBMIT: 'loading' } }, loading: { on: { REJECT: 'error', RESOLVE: 'success' } }, error: { on: { SUBMIT: 'loading' } }, success: {} } }; function transition(state, event) { return machine .states[state] // Look up the state .on[event] // Look up the next state based on the event || state; // If not found, return the current state } console.log(transition('start', 'SUBMIT')); As you might have noticed, the loginMachine is a plain JS object, and can be written in JSON. This is important because it allows the machine to be visualized by a 3rd-party tool, as demonstrated here: A Common Language Between Designers and Developers Although finite state machines are a fundamental part of computer science, they have an amazing potential to bridge the application specification gap between designers and developers, as well as project managers, stakeholders, and more. By designing a state machine visually and with code, designers and developers alike can: identify all possible states, and potentially missing states describe exactly what should happen when an event occurs on a given state, and prevent that event from having unintended side-effects in other states (ever click a submit button more than once?) eliminate impossible states and identify states that are “unreachable” (have no entry transition) or “sunken” (have no exit transition) add features with full confidence of knowing what other states it might affect simplify redundant states or complex user flows create test paths for almost every possible user flow, and easily identify edge cases collaborate better by understanding the entire application model equally. Not a New Idea I’m not the first to suggest that state machines can help bridge the gap between design and development. Vince MingPu Shao wrote an article about designing UI states and communicating with developers effectively with finite state machines User flow diagrams, which visually describe the paths that a user can take through an app to achieve certain goals, are essentially state machines. Numerous tools, from Sketch plugins to standalone apps, exist for creating them. In 1999, Ian Horrocks wrote a book titled “Constructing the User Interface with Statecharts”, which takes state machines to the next level and describes the inherent difficulties (and solutions) with creating complex UIs. The ideas in the book are still relevant today. More than a decade earlier, David Harel published “Statecharts: A Visual Formalism for Complex Systems”, in which the statechart - an extended hierarchical state machine model - is born. State machines and statecharts have been used for complex systems and user interfaces, both physical and digital, for decades, and are especially prevalent in other industries, such as game development and embedded electronic systems. Even NASA uses statecharts for the Curiosity Rover and more, citing many benefits: Visualized modeling Precise diagrams Automatic code generation Comprehensive test coverage Accommodation of late-breaking requirements changes Moving Forward It’s time that we improve how we communicate between designers and developers, much less improve the way we develop UIs to deliver the best, bug-free, optimal user experience. There is so much more to state machines and statecharts than just being a different way of designing and coding. For more resources: The World of Statecharts is a comprehensive guide by Erik Mogensen in using statecharts in your applications The Statechart Community on Spectrum is always full of interesting ideas and questions related to state machines, statecharts, and software modeling I gave a talk at React Rally over a year ago about how state machines (finite automata) can improve the way we develop applications. The latest one is from Reactive Conf, where I demonstrate how statecharts can be used to automatically generate test cases. I have also been working on XState, which is a library for “state machines and statecharts for the modern web”. You can create and visualize statecharts in JavaScript, and use them in any framework (and soon enough, multiple different languages). I’m excited about the future of developing web and mobile applications with statecharts, especially with regard to faster design/development cycles, auto-generated testing, better error prevention, comprehensive analytics, and even the use of model-based reinforcement learning and artificial intelligence to greatly improve the user experience.",2018,David Khourshid,davidkhourshid,2018-12-12T00:00:00+00:00,https://24ways.org/2018/state-machines-in-user-interfaces/,code 258,Mistletoe Offline,"It’s that time of year, when we gather together as families to celebrate the life of the greatest person in history. This man walked the Earth long before us, but he left behind words of wisdom. Those words can guide us every single day, but they are at the forefront of our minds during this special season. I am, of course, talking about Murphy, and the golden rule he gave unto us: Anything that can go wrong will go wrong. So true! I mean, that’s why we make sure we’ve got nice 404 pages. It’s not that we want people to ever get served a File Not Found message, but we acknowledge that, despite our best efforts, it’s bound to happen sometime. Murphy’s Law, innit? But there are some Murphyesque situations where even your lovingly crafted 404 page won’t help. What if your web server is down? What if someone is trying to reach your site but they lose their internet connection? These are all things than can—and will—go wrong. I guess there’s nothing we can do about those particular situations, right? Wrong! A service worker is a Murphy-battling technology that you can inject into a visitor’s device from your website. Once it’s installed, it can intercept any requests made to your domain. If anything goes wrong with a request—as is inevitable—you can provide instructions for the browser. That’s your opportunity to turn those server outage frowns upside down. Take those network connection lemons and make network connection lemonade. If you’ve got a custom 404 page, why not make a custom offline page too? Get your server in order Step one is to make …actually, wait. There’s a step before that. Step zero. Get your site running on HTTPS, if it isn’t already. You won’t be able to use a service worker unless everything’s being served over HTTPS, which makes sense when you consider the awesome power that a service worker wields. If you’re developing locally, service workers will work fine for localhost, even without HTTPS. But for a live site, HTTPS is a must. Make an offline page Alright, assuming your site is being served over HTTPS, then step one is to create an offline page. Make it as serious or as quirky as is appropriate for your particular brand. If the website is for a restaurant, maybe you could put the telephone number and address of the restaurant on the custom offline page (unsolicited advice: you could also put this on the home page, you know). Here’s an example of the custom offline page for this year’s Ampersand conference. When you’re done, publish the offline page at suitably imaginative URL, like, say /offline.html. Pre-cache your offline page Now create a JavaScript file called serviceworker.js. This is the script that the browser will look to when certain events are triggered. The first event to handle is what to do when the service worker is installed on the user’s device. When that happens, an event called install is fired. You can listen out for this event using addEventListener: addEventListener('install', installEvent => { // put your instructions here. }); // end addEventListener In this case, you want to make sure that your lovingly crafted custom offline page is put into a nice safe cache. You can use the Cache API to do this. You get to create as many caches as you like, and you can call them whatever you want. Here, I’m going to call the cache Johnny just so I can refer to it as JohnnyCache in the code: addEventListener('install', installEvent => { installEvent.waitUntil( caches.open('Johnny') .then( JohnnyCache => { JohnnyCache.addAll([ '/offline.html' ]); // end addAll }) // end open.then ); // end waitUntil }); // end addEventListener I’m betting that your lovely offline page is linking to a CSS file, maybe an image or two, and perhaps some JavaScript. You can cache all of those at this point: addEventListener('install', installEvent => { installEvent.waitUntil( caches.open('Johnny') .then( JohnnyCache => { JohnnyCache.addAll([ '/offline.html', '/path/to/stylesheet.css', '/path/to/javascript.js', '/path/to/image.jpg' ]); // end addAll }) // end open.then ); // end waitUntil }); // end addEventListener Make sure that the URLs are correct. If just one of the URLs in the list fails to resolve, none of the items in the list will be cached. Intercept requests The next event you want to listen for is the fetch event. This is probably the most powerful—and, let’s be honest, the creepiest—feature of a service worker. Once it has been installed, the service worker lurks on the user’s device, waiting for any requests made to your site. Every time the user requests a web page from your site, a fetch event will fire. Every time that page requests a style sheet or an image, a fetch event will fire. You can provide instructions for what should happen each time: addEventListener('fetch', fetchEvent => { // What happens next is up to you! }); // end addEventListener Let’s write a fairly conservative script with the following logic: Whenever a file is requested, First, try to fetch it from the network, But if that doesn’t work, try to find it in the cache, But if that doesn’t work, and it’s a request for a web page, show the custom offline page instead. Here’s how that translates into JavaScript: // Whenever a file is requested addEventListener('fetch', fetchEvent => { const request = fetchEvent.request; fetchEvent.respondWith( // First, try to fetch it from the network fetch(request) .then( responseFromFetch => { return responseFromFetch; }) // end fetch.then // But if that doesn't work .catch( fetchError => { // try to find it in the cache caches.match(request) .then( responseFromCache => { if (responseFromCache) { return responseFromCache; // But if that doesn't work } else { // and it's a request for a web page if (request.headers.get('Accept').includes('text/html')) { // show the custom offline page instead return caches.match('/offline.html'); } // end if } // end if/else }) // end match.then }) // end fetch.catch ); // end respondWith }); // end addEventListener I am fully aware that I may have done some owl-drawing there. If you need a more detailed breakdown of what’s happening at each point in the code, I’ve written a whole book for you. It’s the perfect present for Murphymas. Hook up your service worker script You can publish your service worker script at /serviceworker.js but you still need to tell the browser where to look for it. You can do that using JavaScript. Put this in an existing JavaScript file that you’re calling in to every page on your site, or add this in a script element at the end of every page’s HTML: if (navigator.serviceWorker) { navigator.serviceWorker.register('/serviceworker.js'); } That tells the browser to start installing the service worker, but not without first checking that the browser understands what a service worker is. When it comes to JavaScript, feature detection is your friend. You might already have some JavaScript files in a folder like /assets/js/ and you might be tempted to put your service worker script in there too. Don’t do that. If you do, the service worker will only be able to handle requests made to for files within /assets/js/. By putting the service worker script in the root directory, you’re making sure that every request can be intercepted. Go further! Nicely done! You’ve made sure that if—no, when—a visitor can’t reach your website, they’ll get your hand-tailored offline page. You have temporarily defeated the forces of chaos! You have briefly fought the tide of entropy! You have made a small but ultimately futile gesture against the inevitable heat-death of the universe! This is just the beginning. You can do more with service workers. What if, every time you fetched a page from the network, you stored a copy of that page in a cache? Then if that person tries to reach that page later, but they’re offline, you could show them the cached version. Or, what if instead of reaching out the network first, you checked to see if a file is in the cache first? You could serve up that cached version—which would be blazingly fast—and still fetch a fresh version from the network in the background to pop in the cache for next time. That might be a good strategy for images. So many options! The hard part isn’t writing the code, it’s figuring out the steps you want to take. Once you’ve got those steps written out, then it’s a matter of translating them into JavaScript. Inevitably there will be some obstacles along the way—usually it’s a misplaced curly brace or a missing parenthesis. Don’t be too hard on yourself if your code doesn’t work at first. That’s just Murphy’s Law in action.",2018,Jeremy Keith,jeremykeith,2018-12-04T00:00:00+00:00,https://24ways.org/2018/mistletoe-offline/,code 260,The Art of Mathematics: A Mandala Maker Tutorial,"In front-end development, there’s often a great deal of focus on tools that aim to make our work more efficient. But what if you’re new to web development? When you’re just starting out, the amount of new material can be overwhelming, particularly if you don’t have a solid background in Computer Science. But the truth is, once you’ve learned a little bit of JavaScript, you can already make some pretty impressive things. A couple of years back, when I was learning to code, I started working on a side project. I wanted to make something colorful and fun to share with my friends. This is what my app looks like these days: Mandala Maker user interface The coolest part about it is the fact that it’s a tool: anyone can use it to create something original and brand new. In this tutorial, we’ll build a smaller version of this app – a symmetrical drawing tool in ES5, JavaScript and HTML5. The tutorial app will have eight reflections, a color picker and a Clear button. Once we’re done, you’re on your own and can tweak it as you please. Be creative! Preparations: a blank canvas The first thing you’ll need for this project is a designated drawing space. We’ll use the HTML5 canvas element and give it a width and a height of 600px (you can set the dimensions to anything else if you like). Files Create 3 files: index.html, styles.css, main.js. Don’t forget to include your JS and CSS files in your HTML.

Your browser doesn't support canvas.

I’ll ask you to update your HTML file at a later point, but the CSS file we’ll start with will stay the same throughout the project. This is the full CSS we are going to use: body { background-color: #ccc; text-align: center; } canvas { touch-action: none; background-color: #fff; } button { font-size: 110%; } Next steps We are done with our preparations and ready to move on to the actual tutorial, which is made up of 4 parts: Building a simple drawing app with one line and one color Adding a Clear button and a color picker Adding more functionality: 2 line drawing (add the first reflection) Adding more functionality: 8 line drawing (add 6 more reflections!) Interactive demos This tutorial will be accompanied by four CodePens, one at the end of each section. In my own app I originally used mouse events, and only added touch events when I realized mobile device support was (A) possible, and (B) going to make my app way more accessible. For the sake of code simplicity, I decided that in this tutorial app I will only use one event type, so I picked a third option: pointer events. These are supported by some desktop browsers and some mobile browsers. An up-to-date version of Chrome is probably your best bet. Part 1: A simple drawing app Let’s get started with our main.js file. Our basic drawing app will be made up of 6 functions: init, drawLine, stopDrawing, recordPointerLocation, handlePointerMove, handlePointerDown. It also has nine variables: var canvas, context, w, h, prevX = 0, currX = 0, prevY = 0, currY = 0, draw = false; The variables canvas and context let us manipulate the canvas. w is the canvas width and h is the canvas height. The four coordinates are used for tracking the current and previous location of the pointer. A short line is drawn between (prevX, prevY) and (currX, currY) repeatedly many times while we move the pointer upon the canvas. For your drawing to appear, three conditions must be met: the pointer (be it a finger, a trackpad or a mouse) must be down, it must be moving and the movement has to be on the canvas. If these three conditions are met, the boolean draw is set to true. 1. init Responsible for canvas set up, this listens to pointer events and the location of their coordinates and sets everything in motion by calling other functions, which in turn handle touch and movement events. function init() { canvas = document.querySelector(""canvas""); context = canvas.getContext(""2d""); w = canvas.width; h = canvas.height; canvas.onpointermove = handlePointerMove; canvas.onpointerdown = handlePointerDown; canvas.onpointerup = stopDrawing; canvas.onpointerout = stopDrawing; } 2. drawLine This is called to action by handlePointerMove() and draws the pointer path. It only runs if draw = true. It uses canvas methods you can read about in the canvas API documentation. You can also learn to use the canvas element in this tutorial. lineWidth and linecap set the properties of our paint brush, or digital pen, but pay attention to beginPath and closePath. Between those two is where the magic happens: moveTo and lineTo take canvas coordinates as arguments and draw from (a,b) to (c,d), which is to say from (prevX,prevY) to (currX,currY). function drawLine() { var a = prevX, b = prevY, c = currX, d = currY; context.lineWidth = 4; context.lineCap = ""round""; context.beginPath(); context.moveTo(a, b); context.lineTo(c, d); context.stroke(); context.closePath(); } 3. stopDrawing This is used by init when the pointer is not down (onpointerup) or is out of bounds (onpointerout). function stopDrawing() { draw = false; } 4. recordPointerLocation This tracks the pointer’s location and stores its coordinates. Also, you need to know that in computer graphics the origin of the coordinate space (0,0) is at the top left corner, and all elements are positioned relative to it. When we use canvas we are dealing with two coordinate spaces: the browser window and the canvas itself. This function converts between the two: it subtracts the canvas offsetLeft and offsetTop so we can later treat the canvas as the only coordinate space. If you are confused, read more about it. function recordPointerLocation(e) { prevX = currX; prevY = currY; currX = e.clientX - canvas.offsetLeft; currY = e.clientY - canvas.offsetTop; } 5. handlePointerMove This is set by init to run when the pointer moves. It checks if draw = true. If so, it calls recordPointerLocation to get the path and drawLine to draw it. function handlePointerMove(e) { if (draw) { recordPointerLocation(e); drawLine(); } } 6. handlePointerDown This is set by init to run when the pointer is down (finger is on touchscreen or mouse it clicked). If it is, calls recordPointerLocation to get the path and sets draw to true. That’s because we only want movement events from handlePointerMove to cause drawing if the pointer is down. function handlePointerDown(e) { recordPointerLocation(e); draw = true; } Finally, we have a working drawing app. But that’s just the beginning! See the Pen Mandala Maker Tutorial: Part 1 by Hagar Shilo (@hagarsh) on CodePen. Part 2: Add a Clear button and a color picker Now we’ll update our HTML file, adding a menu div with an input of the type and class color and a button of the class clear.

Your browser doesn't support canvas.

Color picker This is our new color picker function. It targets the input element by its class and gets its value. function getColor() { return document.querySelector("".color"").value; } Up until now, the app used a default color (black) for the paint brush/digital pen. If we want to change the color we need to use the canvas property strokeStyle. We’ll update drawLine by adding strokeStyle to it and setting it to the input value by calling getColor. function drawLine() { //...code... context.strokeStyle = getColor(); context.lineWidth = 4; context.lineCap = ""round""; //...code... } Clear button This is our new Clear function. It responds to a button click and displays a dialog asking the user if she really wants to delete the drawing. function clearCanvas() { if (confirm(""Want to clear?"")) { context.clearRect(0, 0, w, h); } } The method clearRect takes four arguments. The first two (0,0) mark the origin, which is actually the top left corner of the canvas. The other two (w,h) mark the full width and height of the canvas. This means the entire canvas will be erased, from the top left corner to the bottom right corner. If we were to give clearRect a slightly different set of arguments, say (0,0,w/2,h), the result would be different. In this case, only the left side of the canvas would clear up. Let’s add this event handler to init: function init() { //...code... canvas.onpointermove = handleMouseMove; canvas.onpointerdown = handleMouseDown; canvas.onpointerup = stopDrawing; canvas.onpointerout = stopDrawing; document.querySelector("".clear"").onclick = clearCanvas; } See the Pen Mandala Maker Tutorial: Part 2 by Hagar Shilo (@hagarsh) on CodePen. Part 3: Draw with 2 lines It’s time to make a line appear where no pointer has gone before. A ghost line! For that we are going to need four new coordinates: a', b', c' and d' (marked in the code as a_, b_, c_ and d_). In order for us to be able to add the first reflection, first we must decide if it’s going to go over the y-axis or the x-axis. Since this is an arbitrary decision, it doesn’t matter which one we choose. Let’s go with the x-axis. Here is a sketch to help you grasp the mathematics of reflecting a point across the x-axis. The coordinate space in my sketch is different from my explanation earlier about the way the coordinate space works in computer graphics (more about that in a bit!). Now, look at A. It shows a point drawn where the pointer hits, and B shows the additional point we want to appear: a reflection of the point across the x-axis. This is our goal. A sketch illustrating the mathematics of reflecting a point. What happens to the x coordinates? The variables a/a' and c/c' correspond to prevX and currX respectively, so we can call them “the x coordinates”. We are reflecting across x, so their values remain the same, and therefore a' = a and c' = c. What happens to the y coordinates? What about b' and d'? Those are the ones that have to change, but in what way? Thanks to the slightly misleading sketch I showed you just now (of A and B), you probably think that the y coordinates b' and d' should get the negative values of b and d respectively, but nope. This is computer graphics, remember? The origin is at the top left corner and not at the canvas center, and therefore we get the following values: b = h - b, d' = h - d, where h is the canvas height. This is the new code for the app’s variables and the two lines: the one that fills the pointer’s path and the one mirroring it across the x-axis. function drawLine() { var a = prevX, a_ = a, b = prevY, b_ = h-b, c = currX, c_ = c, d = currY, d_ = h-d; //... code ... // Draw line #1, at the pointer's location context.moveTo(a, b); context.lineTo(c, d); // Draw line #2, mirroring the line #1 context.moveTo(a_, b_); context.lineTo(c_, d_); //... code ... } In case this was too abstract for you, let’s look at some actual numbers to see how this works. Let’s say we have a tiny canvas of w = h = 10. Now let a = 3, b = 2, c = 4 and d = 3. So b' = 10 - 2 = 8 and d' = 10 - 3 = 7. We use the top and the left as references. For the y coordinates this means we count from the top, and 8 from the top is also 2 from the bottom. Similarly, 7 from the top is 3 from the bottom of the canvas. That’s it, really. This is how the single point, and a line (not necessarily a straight one, by the way) is made up of many, many small segments that are similar to point in behavior. If you are still confused, I don’t blame you. Here is the result. Draw something and see what happens. See the Pen Mandala Maker Tutorial: Part 3 by Hagar Shilo (@hagarsh) on CodePen. Part 4: Draw with 8 lines I have made yet another confusing sketch, with points C and D, so you understand what we’re trying to do. Later on we’ll look at points E, F, G and H as well. The circled point is the one we’re adding at each particular step. The circled point at C has the coordinates (-3,2) and the circled point at D has the coordinates (-3,-2). Once again, keep in mind that the origin in the sketches is not the same as the origin of the canvas. A sketch illustrating points C and D. This is the part where the math gets a bit mathier, as our drawLine function evolves further. We’ll keep using the four new coordinates: a', b', c' and d', and reassign their values for each new location/line. Let’s add two more lines in two new locations on the canvas. Their locations relative to the first two lines are exactly what you see in the sketch above, though the calculation required is different (because of the origin points being different). function drawLine() { //... code ... // Reassign values a_ = w-a; b_ = b; c_ = w-c; d_ = d; // Draw the 3rd line context.moveTo(a_, b_); context.lineTo(c_, d_); // Reassign values a_ = w-a; b_ = h-b; c_ = w-c; d_ = h-d; // Draw the 4th line context.moveTo(a_, b_); context.lineTo(c_, d_); //... code ... What is happening? You might be wondering why we use w and h as separate variables, even though we know they have the same value. Why complicate the code this way for no apparent reason? That’s because we want the symmetry to hold for a rectangular canvas as well, and this way it will. Also, you may have noticed that the values of a' and c' are not reassigned when the fourth line is created. Why write their value assignments twice? It’s for readability, documentation and communication. Maintaining the quadruple structure in the code is meant to help you remember that all the while we are dealing with two y coordinates (current and previous) and two x coordinates (current and previous). What happens to the x coordinates? As you recall, our x coordinates are a (prevX) and c (currX). For the third line we are adding, a' = w - a and c' = w - c, which means… For the fourth line, the same thing happens to our x coordinates a and c. What happens to the y coordinates? As you recall, our y coordinates are b (prevY) and d (currY). For the third line we are adding, b' = b and d' = d, which means the y coordinates are the ones not changing this time, making this is a reflection across the y-axis. For the fourth line, b' = h - b and d' = h - d, which we’ve seen before: that’s a reflection across the x-axis. We have four more lines, or locations, to define. Note: the part of the code that’s responsible for drawing a micro-line between the newly calculated coordinates is always the same: context.moveTo(a_, b_); context.lineTo(c_, d_); We can leave it out of the next code snippets and just focus on the calculations, i.e, the reassignments. Once again, we need some concrete examples to see where we’re going, so here’s another sketch! The circled point E has the coordinates (2,3) and the circled point F has the coordinates (2,-3). The ability to draw at A but also make the drawing appear at E and F (in addition to B, C and D that we already dealt with) is the functionality we are about to add to out code. A sketch illustrating points E and F. This is the code for E and F: // Reassign for 5 a_ = w/2+h/2-b; b_ = w/2+h/2-a; c_ = w/2+h/2-d; d_ = w/2+h/2-c; // Reassign for 6 a_ = w/2+h/2-b; b_ = h/2-w/2+a; c_ = w/2+h/2-d; d_ = h/2-w/2+c; Their x coordinates are identical and their y coordinates are reversed to one another. This one will be out final sketch. The circled point G has the coordinates (-2,3) and the circled point H has the coordinates (-2,-3). A sketch illustrating points G and H. This is the code: // Reassign for 7 a_ = w/2-h/2+b; b_ = w/2+h/2-a; c_ = w/2-h/2+d; d_ = w/2+h/2-c; // Reassign for 8 a_ = w/2-h/2+b; b_ = h/2-w/2+a; c_ = w/2-h/2+d; d_ = h/2-w/2+c; //...code... } Once again, the x coordinates of these two points are the same, while the y coordinates are different. And once again I won’t go into the full details, since this has been a long enough journey as it is, and I think we’ve covered all the important principles. But feel free to play around with the code and change it. I really recommend commenting out the code for some of the points to see what your drawing looks like without them. I hope you had fun learning! This is our final app: See the Pen Mandala Maker Tutorial: Part 4 by Hagar Shilo (@hagarsh) on CodePen.",2018,Hagar Shilo,hagarshilo,2018-12-02T00:00:00+00:00,https://24ways.org/2018/the-art-of-mathematics/,code 263,Securing Your Site like It’s 1999,"Running a website in the early years of the web was a scary business. The web was an evolving medium, and people were finding new uses for it almost every day. From book stores to online auctions, the web was an expanding universe of new possibilities. As the web evolved, so too did the knowledge of its inherent security vulnerabilities. Clever tricks that were played on one site could be copied on literally hundreds of other sites. It was a normal sight to log in to a website to find nothing working because someone had breached its defences and deleted its database. Lessons in web security in those days were hard-earned. What follows are examples of critical mistakes that brought down several early websites, and how you can help protect yourself and your team from the same fate. Bad input validation: Trusting anything the user sends you Our story begins in the most unlikely place: Animal Crossing. Animal Crossing was a 2001 video game set in a quaint town, filled with happy-go-lucky inhabitants that co-exist peacefully. Like most video games, Animal Crossing was the subject of many fan communities on the early web. One such unofficial web forum was dedicated to players discussing their adventures in Animal Crossing. Players could trade secrets, ask for help, and share pictures of their virtual homes. This might sound like a model community to you, but you would be wrong. One day, a player discovered a hidden field in the forum’s user profile form. Normally, this page allows users to change their name, their password, or their profile photo. This person discovered that the hidden field contained their unique user ID, which identifies them when the forum’s backend saves profile changes to its database. They discovered that by modifying the form to change the user ID, they could make changes to any other player’s profile. Needless to say, this idyllic online community descended into chaos. Users changed each other’s passwords, deleted each other’s messages, and attacked each-other under the cover of complete anonymity. What happened? There aren’t any official rules for developing software on the web. But if there were, my golden rule would be: Never trust user input. Ever. Always ask yourself how users will send you data that isn’t what it seems to be. If the nicest community of gamers playing the happiest game on earth can turn on each other, nowhere on the web is safe. Make sure you validate user input to make sure it’s of the correct type (e.g. string, number, JSON string) and that it’s the length that you were expecting. Don’t forget that user input doesn’t become safe once it is stored in your database; any data that originates from outside your network can still be dangerous and must be escaped before it is inserted into HTML. Make sure to check a user’s actions against what they are allowed to do. Create a clear access control policy that defines what actions a user may take, and to whose data they are allowed access to. For example, a newly-registered user should not be allowed to change the user profile of a web forum’s owner. Finally, never rely on client-side validation. Validating user input in the browser is a convenience to the user, not a security measure. Always assume the user has full control over any data sent from the browser and make sure you validate any data sent to your backend from the outside world. SQL injection: Allowing the user to run their own database queries A long time ago, my favourite website was a web forum dedicated to the Final Fantasy video game series. Like the users of the Animal Crossing forum, I’d while away many hours arguing with other people on the internet about my favourite characters, my favourite stories, and the greatest controversies of the day. One day, I noticed people were acting strangely. Users were being uncharacteristically nasty and posting in private areas of the forum they wouldn’t normally have access to. Then messages started disappearing, and user accounts for well-respected people were banned. It turns out someone had discovered a way of logging in to any other user account, using a secret password that allowed them to do literally anything they wanted. What was this password that granted untold power to those who wielded it? ' OR '1'='1 SQL is a computer language that is used to query databases. When you fill out a login form, just like the one above, your username and your password are usually inserted into an SQL query like this: SELECT COUNT(*) FROM USERS WHERE USERNAME='Alice' AND PASSWORD='hunter2' This query selects users from the database that match the username Alice and the password hunter2. If there is at least one user matching record, the user will be granted access. Let’s see what happens when we use our magic password instead! SELECT COUNT(*) FROM USERS WHERE USERNAME='Admin' AND PASSWORD='' OR '1'='1' Does the password look like part of the query to you? That’s because it is! This password is a deliberate attempt to inject our own SQL into the query, hence the term SQL injection. The query is now looking for users matching the username Admin, with a password that is blank, or 1=1. In an SQL query, 1=1 is always true, which makes this query select every single record in the database. As long as the forum software is checking for at least one matching user, it will grant the person logging in access. This password will work for any user registered on the forum! So how can you protect yourself from SQL injection? Never build SQL queries by concatenating strings. Instead, use parameterised query tools. PHP offers prepared statements, and Node.JS has the knex package. Alternatively, you can use an ORM tool, such as Propel or sequelize. Expert help in the form of language features or software tools is a key ally for securing your code. Get all the help you can! Cross site request forgery: Getting other users to do your dirty work for you Do you remember Netflix? Not the Netflix we have now, the Netflix that used to rent you DVDs by mailing them to you. My next story is about how someone managed to convince Netflix users to send him their DVDs - free of charge. Have you ever clicked on a hyperlink, only to find something that you weren’t expecting? If you were lucky, you might have just gotten Rickrolled. If you were unlucky… Let’s just say there are older and fouler things than Rick Astley in the dark places of the web. What if you could convince people to visit a page you controlled? And what if those people were Netflix users, and they were logged in? In 2006, Dave Ferguson did just that. He created a harmless-looking page with an image on it: Did you notice the source URL of the image? It’s deliberately crafted to add a particular DVD to your queue. Sprinkle in a few more requests to change the user’s name and shipping address, and you could ship yourself DVDs completely free of charge! This attack is possible when websites unconditionally trust a user’s session cookies without checking where HTTP requests come from. The first check you can make is to verify that a request’s origin and referer headers match the location of the website. These headers can’t be programmatically set. Another check you can use is to add CSRF tokens to your web forms, to verify requests have come from an actual form on your website. Tokens are long, unpredictable, unique strings that are generated by your server and inserted into web forms. When users complete a form, the form data sent to the server can be checked for a recently generated token. This is an effective deterrent of CSRF attacks because CSRF tokens aren’t stored in cookies. You can also set SameSite=Strict when setting cookies with the Set-Cookie HTTP header. This communicates to browsers that cookies are not to be sent with cross-site requests. This is a relatively new feature, though it is well supported in evergreen browsers. Cross site scripting: Someone else’s code running on your website In 2005, Samy Kamkar became famous for having lots of friends. Lots and lots of friends. Samy enjoyed using MySpace which, at the time, was the world’s largest social network. Social networks at that time were more limited than today. For instance, MySpace let you upload photos to your photo gallery, but capped the limit at twelve. Twelve photos. At least you didn’t have to wade through photos of avocado toast back then… Samy discovered that MySpace also locked down the kinds of content that you could post on your MySpace page. He discovered he could inject and
tags into his headline, but ', '' ]; const filterResult = result => { if (ignoreErrors.indexOf(result.context) > -1) { return false; } return true; }; Initially we wanted to focus on fixing the major problems, so we added a rule to ignore notices and warnings. This made the list or errors much smaller and allowed us focus on fixing major issues such as colour contrast and missing alt text. The ignored notices and warnings can be added in later after these larger issues have been resolved. const test = pa11y({ ignore: [ 'notice', 'warning' ], ... }); Jenkins gotchas While using Jenkins we encountered a few problems. Sometimes Jenkins would indicate a build had passed when in reality it had failed. This was because Pa11y had timed out due to PhantomJS throwing an error, or the test didn’t go past the first URL. Pa11y has recently released a new beta version that uses headless Chrome instead of PhantomJS, so hopefully these issues will less occur less often. We tried a few approaches to solve these issues. First we added error handling, iterating over the array of test URLs so that if an unexpected error happened, we could catch it and exit the process with an error indicating that the job had failed (using process.exit(1)). for (const url of urls) { try { console.log(url); let urlResult = await run(url); urlResult = urlResult.filter(filterResult); urlResult.forEach(result => console.log(result)); } catch (e) { console.log('Error:', e); process.exit(1); } } We also had issues with unhandled rejections sometimes caused by a session disconnecting or similar errors. To avoid Jenkins indicating our site was passing with 100% accessibility, when in reality it had not executed any tests, we instructed Jenkins to fail the job when an unhandled rejection or uncaught exception occurred: process.on('unhandledRejection', (reason, p) => { console.log('Unhandled Rejection at:', p, 'reason:', reason); process.exit(1); }); process.on('uncaughtException', (err) => { console.log('Caught exception: ${err}n'); process.exit(1); }); Now it’s your turn That’s it! That’s how we automated accessibility testing for Elsevier ecommerce pages, allowing us to improve our site and make it more accessible for everyone. I hope our experience can help you automate accessibility tests on your own site, and bring the web a step closer to being accessible to all.",2017,Seren Davies,serendavies,2017-12-07T00:00:00+00:00,https://24ways.org/2017/automating-your-accessibility-tests/,code 212,Refactoring Your Way to a Design System,"I love refactoring code. Absolutely love it. There’s something about taking a piece of UI or a bit of code and reworking it in a way that is simpler, modular, and reusable that makes me incredibly happy. I also love design systems work. It gives hybrids like me a home. It seems like everyone is talking about design systems right now. Design systems teams are perfect for those who enjoy doing architectural work and who straddle the line between designer and developer. Una Kravets recently identified some of the reasons that design systems fail, and chief among them are lack of buy-in, underlying architecture, and communication. While it’s definitely easier to establish these before project work begins, that doesn’t mean it is the only path to success. It’s a privilege to work on a greenfield project, and one that is not afforded to many. Companies with complex and/or legacy codebases may not be able to support a full rewrite of their product. In addition, many people feel overwhelmed at the thought of creating a complete system and are at a loss of how or where to even begin the process. This is where refactoring comes into play. According to Martin Fowler, “refactoring is the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure.” It’s largely invisible work, and if you do it right, the end user will never know the difference. What it will do is provide a decent foundation to begin more systematic work. Build a solid foundation When I was first asked to create Pantsuit, the design system for Hillary for America, I was tasked with changing our codebase to be more modular and scalable, without changing the behavior or visual design of the UI. We needed a system in place that would allow for the rapid creation of new projects while maintaining a consistent visual language. In essence, I was asked to refactor our code into a design system. During that refactor, I focused the majority of my efforts on creating a scalable architecture based on the UI components in a single workflow. Since I needed to maintain a 1:1 parity with production, the only changes I could create were under-the-hood. I started with writing coding standards and deciding on a CSS architecture that I would then use as I rewrote sections of the codebase. If you already have these in place, great! If not, then this is an excellent place to start. Even if your dream of a design system is never fully realized, having a coding philosophy and architecture in place will still have far-reaching benefits and implications. I want to note that if your refactor includes creating new coding standards or a CSS architecture, don’t try to switch everything over right away. Instead, focus on a single new feature and isolate/encapsulate your work from the rest of the codebase. Focus on the features The key principle to cleaning up a complex codebase is to always refactor in the service of a feature. — Max Kanat-Alexander Refactoring for the sake of refactoring can easily lead to accusations of misused time and lack of results. Another danger of refactoring is that it can turn into yak-shaving if you aren’t disciplined in your approach. To that end, tying your refactored components to feature work is a great way to limit scope and reduce the rest of unintended changes. For example, the initial work on Pantsuit focused only on components related to the donations flow. Every line of code I wrote was in service to improving the maintainability and modularity of that UI. Because we didn’t have any standards in place, I started with those. From there, I identified all the components present in every step of the donations flow, which included some type styles, buttons, form inputs and error states. Then came the refactor of each individual component. Finally, I reintegrated the newly refactored components into the existing donations flow and tested it against production, checking for visual and behavioral diffs. At the end of this process, I had the beginning of a design system that would grow to serve over 50 applications, and a case study to demonstrate its effectiveness. Ideally, you’ll want to get buy-in from your stakeholders and product owners before you begin any design systems work. However, in the absence of buy-in, linking your work to new feature development is a good way to both limit the scope of your refactor and jump start component creation. In addition, if you’re still trying to convince your team of the benefits of a design system, starting small and using the newly refactored, feature-driven work as a case study is one way showcase a design systems’ value. By providing a concrete example of how working towards a design system contributed to the project’s success, you’re gathering the data necessary to secure buy-in for a larger-scale effort. It’s a great way to show value, rather than just talking about it. Show, don’t tell Perhaps the most important thing you can do for any design system is to document it. The key is to create a frictionless way to keep the documentation up-to-date, otherwise no one will contribute to it, and in turn, it will become obsolete and useless. There are lots of tools out there to help you get started documenting your new system. One of my favorites is KSS, which parses comments in the code and uses them to generate a style guide. For Pantsuit, I used the node version of KSS, along with a template to quickly spin up a documentation site. I’ve listed just a few tools below; for even more, check out the tools sections of styleguides.io. Fractal Pattern Lab Drizzle Fabricator Astrum Catalog Regardless of what tool you settle on, it needs to integrate well with your current workflow. Conclusion: always be refactoring If you’re not lucky enough to be able to start a new design system from scratch, you can start small and work on a single feature or component. With each new project comes a new opportunity to flesh out a new part of the system, and another potential case study to secure buy-in and showcase its value. Make sure to carefully and thoroughly document each new portion of the system as it’s built. After a few projects, you’ll find yourself with a decent start to a design system. Good luck, and happy holidays! Further reading: Why Design Systems Fail CSS Architecture for Design Systems Refactoring: Improving the Design of Existing Code Refactoring CSS: The Three I’s Refactoring is About Features",2017,Mina Markham,minamarkham,2017-12-23T00:00:00+00:00,https://24ways.org/2017/refactoring-your-way-to-a-design-system/,code 213,Accessibility Through Semantic HTML,"Working on Better, a tracker blocker, I spend an awful lot of my time with my nose in other people’s page sources. I’m mostly there looking for harmful tracking scripts, but often notice the HTML on some of the world’s most popular sites is in a sad state of neglect. What does neglected HTML look like? Here’s an example of the markup I found on a news site just yesterday. There’s a bit of text, a few links, and a few images. But mostly it’s div elements.
Some text more text
divs and spans, why do we use them so much? While I find tracking scripts completely inexcusable, I do understand why people write HTML like the above. As developers, we like to use divs and spans as they’re generic elements. They come with no associated default browser styles or behaviour except that div displays as a block, and span displays inline. If we make our page up out of divs and spans, we know we’ll have absolute control over styles and behaviour cross-browser, and we won’t need a CSS reset. Absolute control may seem like an advantage, but there’s a greater benefit to less generic, more semantic elements. Browsers render semantic elements with their own distinct styles and behaviours. For example, button looks and behaves differently from a. And ul is different from ol. These defaults are shortcuts to a more usable and accessible web. They provide consistent and well-tested components for common interactions. Semantic elements aid usability A good example of how browser defaults can benefit the usability of an element is in the as a popover-style menu. On a touchscreen, Safari overlays the same menu over the lower half of the screen as a “picker view.” Option menu in Safari on macOS. Option menu picker in Safari on iOS. The iOS picker is a much better experience than struggling to pick from a complicated interface inside the page. The menu is shown more clearly than in the confined space on the page, which makes the options easier to read. The required swipe and tap gestures are consistent with the rest of the operating system, making the expected interaction easier to understand. The whole menu is scaled up, meaning the gestures don’t need such fine motor control. Good usability is good accessibility. When we choose to use a div or span over a more semantic HTML element, we’re also doing hard work the browser could be doing for us. We don’t need to tie ourselves in knots making a custom div into a keyboard navigable option menu. Using select passes the bulk of the responsibility over to the browser.  Letting the browser do most of the work is also more future-friendly. More devices, with different expected interactions, will be released in the future. When that happens, the devices’ browsers can adapt our sites according to those interactions. Then we can spend our time doing something more fun than rewriting cross-browser JavaScript for each new device. HTML’s impact on accessibility Assistive technology also uses semantic HTML to understand how best to convey each element to its user. For screen readers Semantic HTML gives context to screen readers. Screen readers are a type of assistive technology that reads the content of the screen to the person using it. All sites have a linear page source. Sighted visitors can use visual cues on the page to navigate to their desired content in a non-linear fashion. As screen readers output audio (and sometimes braille), those visual cues aren’t usable in the same way. Screen readers provide alternative means of navigation, enabling people to jump between different types of content, such as links, forms, headings, lists, and paragraphs. If all our content is marked up using divs and spans, we’re not giving screen readers a chance to index the valuable content. For keyboard navigation Keyboard-only navigation is also aided by semantic HTML. Forms, option menus, navigation, video, and audio are particularly hard for people relying on a keyboard to access. For instance, option menus and navigation can be very fiddly if you need to use a mouse to hover a menu open and move to select the desired item at the same time.  Again, we can leave much of the interaction to the browser through semantic HTML. Semantic form elements can convey if a check box has been checked, or which label is associated with which input field. These default behaviours can make the difference between a person being able to use a form or leaving the site out of frustration. Did I convince you yet? I hope so. Let’s finish with some easy guidelines to follow. 1. Use the most semantic HTML element for the job When you reach for a div, first check if there’s a better element to do the job. What is the role of that element? How should a person be interacting with the element? Are you using class names like nav, header, or main? There are HTML5 elements for those sections! Using specific elements can also make writing CSS simpler, and ensure a consistent design with minimal effort. 2. Separate structure and style Don’t choose HTML elements based on how they’re styled in your CSS. Nowadays, common practice is to use class names rather than elements for CSS selectors. You’re unlikely to wrap all your page content in an

element because you want all the text to be big and bold. Still, it can be easy to choose an HTML element because it will be the easiest to style. Focusing on content without style will help us choose the most semantic HTML element without that temptation. For example, you could add a class of .btn to a div to make it look like a button. But we all know that only a button will really behave like a button. 3. Use progressive enhancement for enhanced functionality Airbnb and Groupon recently proved we’re not past the laziness of “this site only works in X browser.” Baffling disregard for the open web aside, making complex interactive experiences work cross-browser and cross-device is not easy. We can use progressive enhancement to layer fancy or unsupported features on top of a baseline “it works” experience.  We should build the baseline experience on a foundation of accessible, semantic HTML. Then, if you really want to add a specific feature for a proprietary browser, you can layer that on top, without breaking the underlying experience. 4. Test your work Validators are always valuable for checking the browser will be able to correctly interpret your markup. Document outline checkers can be valuable for testing your structure, but be aware that the HTML5 document outline is not actually implemented in browsers. Once you’ve got something resembling a web page, test the experience! Ensure that semantic HTML element you chose looks and behaves in a predictable manner consistent with its use across the web. Test cross-browser, test cross-device, and test with assistive technology. Testing with assistive technology is not as expensive as it used to be, you can even use your smartphone for testing on iOS and Android. Your visitors will thank you! Further reading Accessibility For Everyone by Laura Kalbag HTML5 Doctor HTML5 Accessibility An overview of HTML5 Semantics HTML reference on MDN  Heydon Pickering’s Inclusive Design Checklist The Paciello Group’s Inclusive Design Principles",2017,Laura Kalbag,laurakalbag,2017-12-15T00:00:00+00:00,https://24ways.org/2017/accessibility-through-semantic-html/,code 214,Christmas Gifts for Your Future Self: Testing the Web Platform,"In the last year I became a CSS specification editor, on a mission to revitalise CSS Multi-column layout. This has involved learning about many things, one of which has been the Web Platform Tests project. In this article, I’m going to share what I’ve learned about testing the web platform. I’m also going to explain why I think you might want to get involved too. Why test? At one time or another it is likely that you have been frustrated by an issue where you wrote some valid CSS, and one browser did one thing with it and another something else entirely. Experiences like this make many web developers feel that browser vendors don’t work together, or they are actively doing things in a different way to one another to the detriment of those of us who use the platform. You’ll be glad to know that isn’t the case, and that the people who work on browsers want things to be consistent just as much as we do. It turns out however that interoperability, which is the official term for “works in all browsers”, is hard. Thanks to web-platform-tests, a test from another browser vendor just found genuine bug in our code before we shipped 😻— Brian Birtles (@brianskold) February 10, 2017 In order for W3C Specifications to move on to become W3C Recommendations we need to have interoperable implementations. 6.2.4 Implementation Experience Implementation experience is required to show that a specification is sufficiently clear, complete, and relevant to market needs, to ensure that independent interoperable implementations of each feature of the specification will be realized. While no exhaustive list of requirements is provided here, when assessing that there is adequate implementation experience the Director will consider (though not be limited to): is each feature of the current specification implemented, and how is this demonstrated? are there independent interoperable implementations of the current specification? are there implementations created by people other than the authors of the specification? are implementations publicly deployed? is there implementation experience at all levels of the specification’s ecosystem (authoring, consuming, publishing…)? are there reports of difficulties or problems with implementation? https://www.w3.org/2017/Process-20170301/#transition-reqs We all want interoperability, achieving interoperability is part of the standards process. The next question is, how do we make sure that we get it? Unimplemented vs uninteroperable implementations Before looking at how we can try to improve interoperability, I’d like to look at the reasons we don’t always meet that aim. There are a couple of reasons why browser X is not doing the same thing as browser Y. The first reason is that browser X has not implemented that feature yet. Relatively small teams of people work on browser engines, and their resources are spread as thinly as those of any company. Behind those browsers are business or organisational goals which may not match our desire for a shiny feature to be made available. There are ways in which we as the web community can help gently encourage implementations - by requesting the feature, by using it so it shows up in usage reports, or writing about it to show interest. However, there will always be some degree of lag based on priorities. A browser not supporting a feature at all, is reasonably easy to deal with these days. We can test for support with Feature Queries, and create sensible fallbacks. What is harder to deal with is when a feature is implemented in different ways by different browsers. In that situation you use the feature, perhaps referring to the specification to ensure that you are writing your CSS correctly. It looks exactly as you expect in one browser and it’s all broken when you test in another. A frequent cause of this kind of issue is that the specification is not well defined in a particular area or that the specification has changed since one or other browser implemented it. CSS specifications are not developed in a darkened room, then presented to browser vendors to implement as a completed document. It would be nice if it worked like that, however the web platform is a gnarly thing. Before we can be sure that a specification is correct, it needs implementing in order that we can get the interoperable implementations I described earlier. A circular process has to happen. Specifications have to be written, browsers have to implement and find the problems, and then the specification has to be revised. Many people reading this will be familiar with how flexbox changed three times in browsers, leaving us with a mess of incompatibilities and the need to use at least two versions of the spec. This story was an example of this circular process, in this case the specification was flagged as experimental using vendor prefixes. We had become used to using vendor prefixes in production and early adopters of flexbox were bitten by this. Today, specifications are developed behind experimental flags as we saw with CSS Grid Layout. Yet there has to come a time when implementations ship, and remove those flags, and it may be that knowingly or unknowingly some interop issues slip through. You will know these interop issues as “browser bugs”, perhaps you have even reported one (thank you!) and none of us want them, so how do we make the platform as robust as possible? How do we ensure we have interoperability? If you were working on a large web application, with several people committing code, it would be very easy for one person to make a change that inadvertently broke some part of the application. They might not realise the fact that their change would cause a problem, due to not having a complete understanding of the entire codebase. To prevent this from happening, it is accepted good practice to write tests as well as code. The tests can then be run before the application is deployed. Unless you start out from the beginning writing tests, and are very good at writing a test for every bit of code, it is likely that some issues do slip through from time to time. When this happens, a good approach is to not only fix the issue but also to write a test that would stop it ever happening again. That way the test suite improves over time and hopefully fewer issues happen. The web platform is essentially a giant, sprawling application, with a huge range of people working on it in different ways. There is therefore plenty of opportunity for issues to creep in, so it seems like having some way of writing tests and automating those tests on browsers would be a good thing. That, is what the Web Platform Tests project has set out to achieve. Web Platform Tests Web Platform Tests is the test suite for the web platform. It is set of tests for all parts of the web platform, which can be run in any browser and the results reported. This article mostly discusses CSS tests, because I work on CSS. You will find that there are tests covering the full platform, and you can look into whichever area you have the most interest and experience in. If we want to create a test suite for a CSS specification then we need to ensure that every feature of the specification has a related test. If a change is made to the spec, and a test committed that reflects that change, then it should be straightforward to run that test against each browser and see if it passes. Currently, at the CSS Working Group, specifications that are at Candidate Recommendation Status should commit at test with every normative change to the spec. To explain the terminology, a normative change is one that changes some of the normative text of a specification - text that contains instructions as to how a browser should render a certain thing. A Candidate Recommendation is the status at which the Working Group officially request implementations of the spec, therefore it is reasonable to assume that any change may cause an interoperability issue. It is usually the case that representatives from all browsers will have discussed the change, so anyone who needs to change code will be aware. In this case the test allows them to check that their change passes and matches everyone else. Tests would also highlight the situation where a change to the spec caused an issue in a browser that otherwise wouldn’t be aware if it. Just as a test suite for your web application should alert a person committing code, that their change will cause a problem elsewhere. Discovering the tests I’ve found that the more I have understood the effort that goes into interoperability, and the reasons why creating an interoperable web is so hard, running into browser issues has become less frustrating. I have somewhere to go, even if all I can do is log the bug. If you are even slightly interested in the subject, go have a poke around wpt.fyi. You can explore the various parts of the web platform and see how many tests have been committed. All the the CSS tests are under the directory /css where you will find each specification. Find a specification you are interested in, and look at the tests. Each test has a link to run it in your own browser to see if it passes. This can be useful in itself, if you are battling with an issue and have reduced it down to something specific, you can go and look to see if there is a test covering that and whether it appears to pass or fail in the browser you are battling with. If it turns out that the test fails, it’s probably not you! A test on the wpt.fyi dashboard Note: In some tests you will come across mention of a font called Ahem. This is a font designed for testing which contains consistent glyphs. You can read about how to use the font and download it here. Contributing to Web Platform Tests You can also become involved with Web Platform Tests. People often ask me how they can become involved in CSS, and I can think of no better way than by writing tests. You need to really understand a feature to accurately come up with a method of testing if it works or not in the different engines. This is not glamorous work, it is however a very useful thing to be involved with. In addition to helping yourself, and developing the sort of deep knowledge of the platform that enables contribution, you will really help the progress of specifications. There are only a very few people writing specs. If those people also have to write and review all of the tests it slows down their work. If you want a better, more interoperable web, and to massively improve your ability to have nerdy conversations about highly specific things, testing is the place to start. A local testing setup Your first stop should be to visit the home of Web Platform Tests. There is some documentation here, which does tend to assume you know about the tests and what you are looking for - having read this article you know as much as I do. To be able to work on tests you will want to: Clone the WPT repo, this is where all the tests are stored Install some tools so you can run up a local copy of the tests The instructions on the Readme in the repo should get you up and running, you can then load your own version of the test suite in a browser at http://web-platform-test:8000, or whichever port you set up. Running tests locally Finding things to test It’s currently not straightforward to locate low-hanging fruit in order to start committing tests. There are some issues flagged up as a good first issue in the GitHub repo, if any of those match your interest and knowledge. Otherwise, a good place to start is where you know of existing interoperability issues. If you are aware of a browser bug, have a look and see if there is a test that addresses it. If not, then a test highlights the interoperability issue, and if it is an issue that you are running into means that you have a nice way to see if it has been fixed! Talk to people There is an IRC channel at irc://irc.w3.org:6667/testing, where you will find people who are writing tests as well as people who are working on the test suite framework itself. They have always been very friendly, and are likely to welcome people with a real interest in creating tests. Gathering information First you need to read the spec. To be able to create a test you need to know and to understand what the specification says should be happening. As I mentioned, writing tests will improve your knowledge dramatically! In general I find that web developers assume their favourite browser has got it right, this isn’t about right or wrong however, or good browsers versus bad ones. The browser with the incorrect implementation may have had a perfect, as per the spec implementation, until something changed. Do some investigation and work out what the spec says, and which – if any – browser is doing it correctly. Another good place to look when trying to create a test for an interop issue, is to look at the browser issue trackers. It is quite likely that someone has already logged the issue, and detailed what it is, and even which browsers are as per the spec. This is useful information, as you then have a clue as to which browsers should pass your test. Remember to check version numbers - an issue may well be fixed in a pre-release version of Chrome for example, but not in the public release. Edge Issue Tracker Mozilla Issue Tracker WebKit Issue Tracker Chromium Issue Tracker Writing the test If you’ve ever created a Reduced Test Case to isolate a browser issue, you already have some idea of what we are trying to do with a test. We want to test one thing, in isolation, and to be able to confirm “yes this works as per the spec” or “no, this does not”. The main two types of test are: testharness.js tests reftests The testharness.js tests use JavaScript to test an assertion, this framework is designed as a way to test Web APIs and as this quickly gets fairly complicated - and I’m a complete beginner myself at writing these - I’ll refer you to the excellent tutorial Using testharness.js. Many CSS tests will be reftests. A reftest involves getting two pages to lay out in the same way, so that they are visually the same. For example, you can find a reftest for Grid Layout at:https://w3c-test.org/css/css-grid/alignment/grid-gutters-001.html or at http://web-platform.test:8000/css/css-grid/alignment/grid-gutters-001.html if you have run up your own copy of WPT. CSS Grid Layout Test: Support for gap shorthand property of row-gap and column-gap

The test passes if it has the same visual effect as reference.

I am testing the new gap property (renamed grid-gap). The reference file can be found by looking for the line: In that file, I am using absolute positioning to mock up the way the file would look if gap is implemented correctly. CSS Grid Layout Reference: a square with a green cross
The tests are compared in an automated way by taking screenshots of the test and reference. These are relatively simple tests to write, you will find the work is not in writing the test however. The work is really in doing the research, and making sure you understand what is supposed to happen so you can write the test. Which is why, if you really want to get your hands dirty in the web platform, this is a good place to start. Committing a test Once you have written a test you can run the lint tool to make sure that everything is tidy. This tool is run automatically after you submit your pull request, and reviewers won’t accept a test with lint errors, so do this locally first to catch anything obvious. Tests are added as a pull request, once you have your test ready to go you can create a pull request to add it to the repository. Your test will be tested and it will then wait for a review. You may well then find yourself in a bit of a waiting game, as the test needs to be reviewed. How long that takes will depend on how active work is on that spec. People who are in the OWNERS file for that spec should be notified. You can always ask in IRC to see if someone is available who can look at and potentially merge your test. Usually the reviewer will have some comments as to how the test can be improved, in the same as the owner of an open source project you submit a PR to might ask you to change some things. Work with them to make your test as good as it can be, the things you learn on the first test you submit will make future ones easier. You can then bask in the glow of knowing you have done something towards the aim of a more interoperable web for all of us. Christmas gifts for your future self I have been a web developer for over 20 years. I have no idea what the web platform will look like in 20 more years, but for as long as I’m working on it I’ll keep on trying to make it better. Making the web more interoperable makes it a better place to be a web developer, storing up some Christmas gifts for my future self, while learning new things as I do so. Resources I rounded up everything I could find on WPT while researching this article. As well as some other links that might be helpful for you. These links are below. Happy testing! Web Platform Tests Using testharness.js IRC Channel irc://irc.w3.org:6667/testing Edge Issue Tracker Mozilla Issue Tracker WebKit Issue Tracker Chromium Issue Tracker Reducing an Issue - guide to created a reduced test case Effectively Using Web Platform Tests: Slides and Video An excellent walkthrough from Lyza Gardner on her working on tests for the HTML specification - Moving Targets: a case study on testing web standards. Improving interop with web-platform-tests: Slides and Video",2017,Rachel Andrew,rachelandrew,2017-12-10T00:00:00+00:00,https://24ways.org/2017/testing-the-web-platform/,code 215,Teach the CLI to Talk Back,"The CLI is a daunting tool. It’s quick, powerful, but it’s also incredibly easy to screw things up in – either with a mistyped command, or a correctly typed command used at the wrong moment. This puts a lot of people off using it, but it doesn’t have to be this way. If you’ve ever interacted with Slack’s Slackbot to set a reminder or ask a question, you’re basically using a command line interface, but it feels more like having a conversation. (My favourite Slack app is Lunch Train which helps with the thankless task of herding colleagues to a particular lunch venue on time.) Same goes with voice-operated assistants like Alexa, Siri and Google Home. There are even games, like Lifeline, where you interact with a stranded astronaut via pseudo SMS, and KOMRAD where you chat with a Soviet AI. I’m not aiming to build an AI here – my aspirations are a little more down to earth. What I’d like is to make the CLI a friendlier, more forgiving, and more intuitive tool for new or reluctant users. I want to teach it to talk back. Interactive command lines in the wild If you’ve used dev tools in the command line, you’ve probably already used an interactive prompt – something that asks you questions and responds based on your answers. Here are some examples: Yeoman If you have Yeoman globally installed, running yo will start a command prompt. The prompt asks you what you’d like to do, and gives you options with how to proceed. Seasoned users will run specific commands for these options rather than go through this prompt, but it’s a nice way to start someone off with using the tool. npm If you’re a Node.js developer, you’re probably familiar with typing npm init to initialise a project. This brings up prompts that will populate a package.json manifest file for that project. The alternative would be to expect the user to craft their own package.json, which is more error-prone since it’s in JSON format, so something as trivial as an extraneous comma can throw an error. Snyk Snyk is a dev tool that checks for known vulnerabilities in your dependencies. Running snyk wizard in the CLI brings up a list of all the known vulnerabilities, and gives you options on how to deal with it – such as patching the issue, applying a fix by upgrading the problematic dependency, or ignoring the issue (you are then prompted for a reason). These decisions get mapped to the manifest and a .snyk file, and committed into the repo so that the settings are the same for everyone who uses that project. I work at Snyk, and running the wizard is what made me think about building my own personal assistant in the command line to help me with some boring, repetitive tasks. Writing your own Something I do a lot is add bookmarks to styleguides.io – I pull down the entire repo, copy and paste a template YAML file, and edit to contents. Sometimes I get it wrong and break the site. So I’ve been putting together a tool to help me add bookmarks. It’s called bookmarkbot – it’s a personal assistant squirrel called Mark who will collect and bury your bookmarks for safekeeping.* *Fortunately, this metaphor also gives me a charming excuse for any situation where bookmarks sometimes get lost – it’s not my poorly-written code, honest, it’s just being realistic because sometimes squirrels forget where they buried things! When you run bookmarkbot, it will ask you for some information, and save that information as a Markdown file in YAML format. For this demo, I’m going to use a Node.js package called inquirer, which is a well supported tool for creating command line prompts. I like it because it has a bunch of different question types; from input, which asks for some text back, confirm which expects a yes/no response, or a list which gives you a set of options to choose from. You can even nest questions, Choose Your Own Adventure style. Prerequisites Node.js npm RubyGems (Only if you want to go as far as serving a static site for your bookmarks, and you want to use Jekyll for it) Disclaimer Bear in mind that this is a really simplified walkthrough. It doesn’t have any error states, and it doesn’t handle the situation where we save a file with the same name. But it gets you in a good place to start building out your tool. Let’s go! Create a new folder wherever you keep your projects, and give it an awesome name (I’ve called mine bookmarks and put it in the Sites directory because I’m unimaginative). Now cd to that directory. cd Sites/bookmarks Let’s use that example I gave earlier, the trusty npm init. npm init Pop in the information you’d like to provide, or hit ENTER to skip through and save the defaults. Your directory should now have a package.json file in it. Now let’s install some of the dependencies we’ll need. npm install --save inquirer npm install --save slugify Next, add the following snippet to your package.json to tell it to run this file when you run npm start. ""scripts"": { … ""start"": ""node index.js"" } That index.js file doesn’t exist yet, so let’s create it in the root of our folder, and add the following: // Packages we need var fs = require('fs'); // Creates our file (part of Node.js so doesn't need installing) var inquirer = require('inquirer'); // The engine for our questions prompt var slugify = require('slugify'); // Will turn a string into a usable filename // The questions var questions = [ { type: 'input', name: 'name', message: 'What is your name?', }, ]; // The questions prompt function askQuestions() { // Ask questions inquirer.prompt(questions).then(answers => { // Things we'll need to generate the output var name = answers.name; // Finished asking questions, show the output console.log('Hello ' + name + '!'); }); } // Kick off the questions prompt askQuestions(); This is just some barebones where we’re including the inquirer package we installed earlier. I’ve stored the questions in a variable, and the askQuestions function will prompt the user for their name, and then print “Hello ” in the console. Enough setup, let’s see some magic. Save the file, go back to the command line and run npm start. Extending what we’ve learnt At the moment, we’re just saving a name to a file, which isn’t really achieving our goal of saving bookmarks. We don’t want our tool to forget our information every time we talk to it – we need to save it somewhere. So I’m going to add a little function to write the output to a file. Saving to a file Create a folder in your project’s directory called _bookmarks. This is where the bookmarks will be saved. I’ve replaced my questions array, and instead of asking for a name, I’ve extended out the questions, asking to be provided with a link and title (as a regular input type), a list of tags (using inquirer’s checkbox type), and finally a description, again, using the input type. So this is how my code looks now: // Packages we need var fs = require('fs'); // Creates our file var inquirer = require('inquirer'); // The engine for our questions prompt var slugify = require('slugify'); // Will turn a string into a usable filename // The questions var questions = [ { type: 'input', name: 'link', message: 'What is the url?', }, { type: 'input', name: 'title', message: 'What is the title?', }, { type: 'checkbox', name: 'tags', message: 'Would you like me to add any tags?', choices: [ { name: 'frontend' }, { name: 'backend' }, { name: 'security' }, { name: 'design' }, { name: 'process' }, { name: 'business' }, ], }, { type: 'input', name: 'description', message: 'How about a description?', }, ]; // The questions prompt function askQuestions() { // Say hello console.log('🐿 Oh, hello! Found something you want me to bookmark?\n'); // Ask questions inquirer.prompt(questions).then((answers) => { // Things we'll need to generate the output var title = answers.title; var link = answers.link; var tags = answers.tags + ''; var description = answers.description; var output = '---\n' + 'title: ""' + title + '""\n' + 'link: ""' + link + '""\n' + 'tags: [' + tags + ']\n' + '---\n' + description + '\n'; // Finished asking questions, show the output console.log('\n🐿 All done! Here is what I\'ve written down:\n'); console.log(output); // Things we'll need to generate the filename var slug = slugify(title); var filename = '_bookmarks/' + slug + '.md'; // Write the file fs.writeFile(filename, output, function () { console.log('\n🐿 Great! I have saved your bookmark to ' + filename); }); }); } // Kick off the questions prompt askQuestions(); The output is formatted into YAML metadata as a Markdown file, which will allow us to turn it into a static HTML file using a build tool later. Run npm start again and have a look at the file it outputs. Getting confirmation Before the user makes critical changes, it’s good to verify those changes first. We’re going to add a confirmation step to our tool, before writing the file. More seasoned CLI users may favour speed over a “hey, can you wait a sec and just check this is all ok” step, but I always think it’s worth adding one so you can occasionally save someone’s butt. So, underneath our questions array, let’s add a confirmation array. // Packages we need … // The questions … // Confirmation questions var confirm = [ { type: 'confirm', name: 'confirm', message: 'Does this look good?', }, ]; // The questions prompt … As we’re adding the confirm step before the file gets written, we’ll need to add the following inside the askQuestions function: // The questions prompt function askQuestions() { // Say hello … // Ask questions inquirer.prompt(questions).then((answers) => { … // Things we'll need to generate the output … // Finished asking questions, show the output … // Confirm output is correct inquirer.prompt(confirm).then(answers => { // Things we'll need to generate the filename var slug = slugify(title); var filename = '_bookmarks/' + slug + '.md'; if (answers.confirm) { // Save output into file fs.writeFile(filename, output, function () { console.log('\n🐿 Great! I have saved your bookmark to ' + filename); }); } else { // Ask the questions again console.log('\n🐿 Oops, let\'s try again!\n'); askQuestions(); } }); }); } // Kick off the questions prompt askQuestions(); Now run npm start and give it a go! Typing y will write the file, and n will take you back to the start. Ideally, I’d store the answers already given as defaults so the user doesn’t have to start from scratch, but I want to keep this demo simple. Serving the files Now that your bookmarking tool is successfully saving formatted Markdown files to a folder, the next step is to serve those files in a way that lets you share them online. The easiest way to do this is to use a static-site generator to convert your YAML files into HTML, and pop them all on one page. Now, you’ve got a few options here and I don’t want to force you down any particular path, as there are plenty out there – it’s just a case of using the one you’re most comfortable with. I personally favour Jekyll because of its tight integration with GitHub Pages – I don’t want to mess around with hosting and deployment, so it’s really handy to have my bookmarks publish themselves on my site as soon as I commit and push them using Git. I’ll give you a very brief run-through of how I’m doing this with bookmarkbot, but I recommend you read my Get Started With GitHub Pages (Plus Bonus Jekyll) guide if you’re unfamiliar with them, because I’ll be glossing over some bits that are already covered in there. Setting up a build tool If you haven’t already, install Jekyll and Bundler globally through RubyGems. Jekyll is our static-site generator, and Bundler is what we use to install Ruby dependencies. gem install jekyll bundler In my project folder, I’m going to run the following which will install the Jekyll files we’ll need to build our listing page. I’m using --force, otherwise it will complain that the directory isn’t empty. jekyll new . --force If you check your project folder, you’ll see a bunch of new files. Now run the following to start the server: bundle exec jekyll serve This will build a new directory called _site. This is where your static HTML files have been generated. Don’t touch anything in this folder because it will get overwritten the next time you build. Now that serve is running, go to http://127.0.0.1:4000/ and you’ll see the default Jekyll page and know that things are set up right. Now, instead, we want to see our list of bookmarks that are saved in the _bookmarks directory (make sure you’ve got a few saved). So let’s get that set up next. Open up the _config.yml file that Jekyll added earlier. In here, we’re going to tell it about our bookmarks. Replace everything in your _config.yml file with the following: title: My Bookmarks description: These are some of my favourite articles about the web. markdown: kramdown baseurl: /bookmarks # This needs to be the same name as whatever you call your repo on GitHub. collections: - bookmarks This will make Jekyll aware of our _bookmarks folder so that we can call it later. Next, create a new directory and file at _layouts/home.html and paste in the following. {{site.title}}

{{site.title}}

{{site.description}}

    {% for bookmark in site.bookmarks %}
  • {{bookmark.title}}

    {{bookmark.content}} {% if bookmark.tags %}
      {% for tags in bookmark.tags %}
    • {{tags}}
    • {% endfor %}
    {% endif %}
  • {% endfor %}
Restart Jekyll for your config changes to kick in, and go to the url it provides you (probably http://127.0.0.1:4000/bookmarks, unless you gave something different as your baseurl). It’s a decent start – there’s a lot more we can do in this area but now we’ve got a nice list of all our bookmarks, let’s get it online! If you want to use GitHub Pages to host your files, your first step is to push your project to GitHub. Go to your repository and click “settings”. Scroll down to the section labelled “GitHub Pages”, and from here you can enable it. Select your master branch, and it will provide you with a url to view your published pages. What next? Now that you’ve got a framework in place for publishing bookmarks, you can really go to town on your listing page and make it your own. First thing you’ll probably want to do is add some CSS, then when you’ve added a bunch of bookmarks, you’ll probably want to have some filtering in place for the tags, perhaps extend the types of questions that you ask to include an image (if you’re feeling extra-fancy, you could just ask for a url and pull in metadata from the site itself). Maybe you’ve got an idea that doesn’t involve bookmarks at all. You could use what you’ve learnt to build a place where you can share quotes, a list of your favourite restaurants, or even Christmas gift ideas. Here’s one I made earlier My demo, bookmarkbot, is on GitHub, and I’ve reused a lot of the code from styleguides.io. Feel free to grab bits of code from there, and do share what you end up making!",2017,Anna Debenham,annadebenham,2017-12-11T00:00:00+00:00,https://24ways.org/2017/teach-the-cli-to-talk-back/,code 216,Styling Components - Typed CSS With Stylable,"There’s been a lot of debate recently about how best to style components for web apps so that styles don’t accidentally ‘leak’ out of the component they’re meant for, or clash with other styles on the page. Elaborate CSS conventions have sprung up, such as OOCSS, SMACSS, BEM, ITCSS, and ECSS. These work well, but they are methodologies, and require everyone in the team to know them and follow them, which can be a difficult undertaking across large or distributed teams. Others just give up on CSS and put all their styles in JavaScript. Now, I’m not bashing JS, especially so close to its 22nd birthday, but CSS-in-JS has problems of its own. Browsers have 20 years experience in optimising their CSS engines, so JavaScript won’t be as fast as using real CSS, and in any case, this requires waiting for JS to download, parse, execute then render the styles. There’s another problem with CSS-in-JS, too. Since Responsive Web Design hit the streets, most designers no longer make comps in Photoshop or its equivalents; instead, they write CSS. Why hire an expensive design professional and require them to learn a new way of doing their job? A recent thread on Twitter asked “What’s your biggest gripe with CSS-in-JS?”, and the replies were illuminating: “Always having to remember to camelCase properties then spending 10min pulling hair out when you do forget”, “the cryptic domain-specific languages that each of the frameworks do just ever so slightly differently”, “When I test look and feel in browser, then I copy paste from inspector, only to have to re-write it as a JSON object”, “Lack of linting, autocomplete, and css plug-ins for colors/ incrementing/ etc”. If you’re a developer, and you’re still unconvinced, I challenge you to let designers change the font in your IDE to Zapf Chancery and choose a new colour scheme, simply because they like it better. Does that sound like fun? Will that boost your productivity? Thought not. Some chums at Wix Engineering and I wanted to see if we could square this circle. Wix-hosted sites have always used CSS-in-JS (the concept isn’t new; it was in Netscape 4!) but that was causing performance problems. Could we somehow devise a method of extending CSS (like SASS and LESS do) that gives us styles that are guaranteed not to leak or clash, that is compatible with code editors’ autocompletion, and which could be pre-processed at build time to valid, cross-browser, static CSS? After a few months and a few proofs of concept later (drumroll), yes – we could! We call it Stylable. Introducing Stylable Stylable is a CSS pre-processor, like SASS or LESS. It uses CSS syntax so all your development tools will work. At build time, the Stylable CSS extensions are transpiled to flat, valid, cross-browser vanilla CSS for maximum performance. There’s quite a bit to it, and this is a short article, so let’s look at the basic concepts. Components all the way down Stylable is designed for component-based systems. Imagine you have a Gallery component. Within that, there is a Navigation component (for example, containing a ‘next’, ‘previous’, ‘show all thumbnails’, and ‘show all albums’ controls), and within that there are NavButton components. Each component is discrete, used elsewhere in the system in different contexts, perhaps maintained by different team members or even different organisations — you can use Stylable to add a typed interface to non-Stylable component libraries, as well as using it to build an app from scratch. Firstly, Stylable will automatically namespace styles so they only apply inside that component, by rewriting them at build time with a unique (but human-readable) prefix. So, for example,
might be re-written as
. So far, so BEM-like (albeit without the headache of remembering a convention). But what else can it do? Custom pseudo-elements An important feature of Stylable is the ability to reach into a component and style it from the outside, without having to know about its internal structure. Let’s see the guts of a simple JSX button component in the file button.jsx: render () { return ( ); } (Note:className is the JSX way of setting a class on an element; this example uses React, but Stylable itself is framework-agnostic.) I style it using a Stylable stylesheet (the .st.css suffix tells the preprocessor to process this file): /* button.st.css */ /* note that the root class is automatically placed on the root HTML element by Stylable React integration */ .root { background: #b0e0e6; } .icon { display: block; height: 2em; background-image: url('./assets/btnIcon.svg'); } .label { font-size: 1.2em; color: rgba(81, 12, 68, 1.0); } Note that Stylable allows all the CSS that you know and love to be included. As Drew Powers wrote in his review: with Stylable, you get CSS, and every part of CSS. This seems like a “duh” observation, but this is significant if you’ve ever battled with a CSS-in-JS framework over a lost or “hacky” implementation of a basic CSS feature. I can import my Button component into another component - this time, panel.jsx: /* panel.jsx */ import * as React from 'react'; import {properties, stylable} from 'wix-react-tools'; import {Button} from '../button'; import style from './panel.st.css'; export const Panel = stylable(style)(() => (
)); In panel.st.css: /* panel.st.css */ :import { -st-from: './button.st.css'; -st-default: Button; } /* cancelBtn is of type Button */ .cancelBtn { -st-extends: Button; background: cornflowerblue; } /* targets the label of