that could have been needed for the nested 14 column grid.
We can move on to the new component.
Creating a new component
Back to the new components we are needing to create.
To me there are two components one of which is a slight variant of the first. This component contains a title, subtitle, a paragraph (potentially paragraphs) of content, a list, and a call to action.
To start with we should write the HTML for the component, something like this:
To place the component on the existing grid is fine, but as child elements are not affected by the container grid we need to define another grid for the features component.
As the grid doesn’t get invoked until 700px it is possible to negate the need for a media query.
.features {
grid-column: col-start 1 / span 6;
}
@supports (display: grid) {
@media (min-width: 1100px) {
.features {
grid-column-end: 9;
}
}
}
We can also avoid duplication of declarations by making use of the grid-column shorthand and longhand. We need to write a little more CSS for the variant component, the one that will sit on the right side of the page too.
.features:nth-of-type(even) {
grid-column-start: 4;
grid-row: 2;
}
@supports (display: grid) {
@media (min-width: 1100px) {
.features:nth-of-type(even) {
grid-column-start: 9;
grid-column-end: 16;
}
}
}
We cannot place the items within features on the container grid as they are not direct children. To make this work we have to define a grid for the features component.
We can do this by defining the grid at the first breakpoint of 700px making use of CSS custom properties again to define how many columns there will need to be.
.features {
grid-column: col-start 1 / span 6;
--features-grid-columns: 5;
}
@supports (display: grid) {
@media (min-width: 700px) {
.features {
display: grid;
grid-gap: .75em;
grid-template-columns: repeat(var(--features-grid-columns), [col-start] 1fr);
}
}
}
@supports (display: grid) {
@media (min-width: 1100px) {
.features {
grid-column-end: 9;
--features-grid-columns: 7;
}
}
}
See the Pen Design Systems and CSS Grid, 10 by Stuart Robson (@sturobson) on CodePen.
Laying out the parts
Looking at the spec and reading several articles I feel there are two ways that I could layout the text of this component on the grid.
We could use the grid-column shorthand that incorporates grid-column-start and grid-column-end or we can make use of grid-template-areas.
grid-template-areas allow for a nice visual way of representing how the parts of the component would be laid out. We can take the the mock of the features on the grid and represent them in text in our CSS.
Within the .features rule we can add the relevant grid-template-areas value to represent the above.
.features {
display: grid;
grid-template-columns: repeat(var(--features-grid-columns), [col-start] 1fr);
grid-template-areas:
"". title title title title title title""
"". subtitle subtitle subtitle subtitle subtitle . ""
"". content content content content . . ""
"". list list list . . . ""
"". . . . link link link "";
}
In order to make the variant of the component we would have to create the grid-template-areas for that component too.
We then need to tell each element of the component in what grid-area it should be placed within the grid.
.features__title { grid-area: title; }
.features__subtitle { grid-area: subtitle; }
.features__content { grid-area: content; }
.features__list { grid-area: list; }
.features__link { grid-area: link; }
See the Pen Design Systems and CSS Grid, 12 by Stuart Robson (@sturobson) on CodePen.
The other way would be to use the grid-column shorthand and the grid-column-start and grid-column-end we have used previously.
.features .features__title {
grid-column: col-start 2 / span 6;
}
.features .features__subtitle {
grid-column: col-start 2 / span 5;
}
.features .features__content {
grid-column: col-start 2 / span 4;
}
.features .features__list {
grid-column: col-start 2 / span 4;
}
.features .features__link {
grid-column: col-start 5 / span 3;
}
For the variant of the component we can use the grid-column-start property as it will inherit the span defined in the grid-column shorthand.
.features:nth-of-type(even) .features__title {
grid-column-start: col-start 1;
}
.features:nth-of-type(even) .features__subtitle {
grid-column-start: col-start 1;
}
.features:nth-of-type(even) .features__content {
grid-column-start: col-start 3;
}
.features:nth-of-type(even) .features__list {
grid-column-start: col-start 3;
}
.features:nth-of-type(even) .features__link {
grid-column-start: col-start 1;
}
See the Pen Design Systems and CSS Grid, 14 by Stuart Robson (@sturobson) on CodePen.
I think, for now, we will go with using grid-column properties rather than grid-template-areas. The repetition needed for creating the variant feels too much where we can change the grid-column-start instead, keeping the components elements layout properties tied a little closer to the elements rather than the grid.
Some additional decisions
The current component library has existing styles for titles, subtitles, lists, paragraphs of text and calls to action. These are name-spaced so that they shouldn’t clash with any other components. Looking forward there will be a chance that other products adopt the component library, but they may bring their own styles for titles, subtitles, etc.
One way that we could write our code now for that near future possibility is to make sure our classes are working hard. Using class-attribute selectors we can target part of the class attributes that we know the elements in the component will have using *=.
.features [class*=""title""] {
grid-column: col-start 2 / span 6;
}
.features [class*=""subtitle""] {
grid-column: col-start 2 / span 5;
}
.features [class*=""content""] {
grid-column: col-start 2 / span 4;
}
.features [class*=""list""] {
grid-column: col-start 2 / span 4;
}
.features [class*=""link""] {
grid-column: col-start 5 / span 3;
}
See the Pen Design Systems and CSS Grid, 15 by Stuart Robson (@sturobson) on CodePen.
Although the component we have created have a title, subtitle, paragraphs, a list, and a call to action there may be a time where one ore more of these is not required or available. One thing I found out is that if the element doesn’t exist then grid will not create space for it. This may be obvious, but it can be really helpful in making a nice malleable component.
We have only looked at columns, as existing components have their own spacing for the vertical rhythm of the page we don’t really want to have them take up equal space in the component and just take up the space as needed. We can do this by adding grid-auto-rows: min-content; to our .features. This is useful if you also need your component to take up a height that is more than the component itself.
The grid of the future
From prototyping this new grid and components in CSS Grid, I’ve found it a fantastic way to reimagine how we can create a layout or grid system for our sites. It gives us options to create the same layouts in differing ways that could suit a project and its needs.
It allows us to carry on – if we choose to – using a
-based grid but swapping out floats for CSS Grid or to tie it to our components so they have specific places to go depending on what component is being used. Or we could have several ‘grid components’ in our design system that we could use to layout various components throughout a page.
If you find yourself tasked with creating some new components for your design system try it. If you are starting from scratch I believe you really should start with CSS Grid for your layout.
It really feels like the possibilities are endless in terms of layout for the web.
Resources
Here are just a few resources I have pawed over these last few weeks whilst getting acquainted with CSS Grid.
A collection of CodePens from this article
Grid by Example from Rachel Andrew
A Complete Guide to CSS Grid on Codrops from Hui Jing Chen
Rachel Andrew’s Blog Archive tagged: cssgrid
CSS Grid Layout Examples
MDN’s CSS Grid Layout
A Complete Guide to Grid from CSS-Tricks
CSS Grid Layout Module Level 1 Specification",2017,Stuart Robson,stuartrobson,2017-12-12T00:00:00+00:00,https://24ways.org/2017/design-systems-and-css-grid/,code
157,Capturing Caps Lock,"One of the more annoying aspects of having to remember passwords (along with having to remember loads of them) is that if you’ve got Caps Lock turned on accidentally when you type one in, it won’t work, and you won’t know why. Most desktop computers alert you in some way if you’re trying to enter your password to log on and you’ve enabled Caps Lock; there’s no reason why the web can’t do the same. What we want is a warning – maybe the user wants Caps Lock on, because maybe their password is in capitals – rather than something that interrupts what they’re doing. Something subtle.
But that doesn’t answer the question of how to do it. Sadly, there’s no way of actually detecting whether Caps Lock is on directly. However, there’s a simple work-around; if the user presses a key, and it’s a capital letter, and they don’t have the Shift key depressed, why then they must have Caps Lock on! Simple.
DOM scripting allows your code to be notified when a key is pressed in an element; when the key is pressed, you get the ASCII code for that key. Capital letters, A to Z, have ASCII codes 65 to 90. So, the code would look something like:
on a key press
if the ASCII code for the key is between 65 and 90 *and* if shift is pressed
warn the user that they have Caps Lock on, but let them carry on
end if
end keypress
The actual JavaScript for this is more complicated, because both event handling and keypress information differ across browsers. Your event handling functions are passed an event object, except in Internet Explorer where you use the global event object; the event object has a which parameter containing the ASCII code for the key pressed, except in Internet Explorer where the event object has a keyCode parameter; some browsers store whether the shift key is pressed in a shiftKey parameter and some in a modifiers parameter. All this boils down to code that looks something like this:
keypress: function(e) {
var ev = e ? e : window.event;
if (!ev) {
return;
}
var targ = ev.target ? ev.target : ev.srcElement;
// get key pressed
var which = -1;
if (ev.which) {
which = ev.which;
} else if (ev.keyCode) {
which = ev.keyCode;
}
// get shift status
var shift_status = false;
if (ev.shiftKey) {
shift_status = ev.shiftKey;
} else if (ev.modifiers) {
shift_status = !!(ev.modifiers & 4);
}
// At this point, you have the ASCII code in “which”,
// and shift_status is true if the shift key is pressed
}
Then it’s just a check to see if the ASCII code is between 65 and 90 and the shift key is pressed. (You also need to do the same work if the ASCII code is between 97 (a) and 122 (z) and the shift key is not pressed, because shifted letters are lower-case if Caps Lock is on.)
if (((which >= 65 && which <= 90) && !shift_status) ||
((which >= 97 && which <= 122) && shift_status)) {
// uppercase, no shift key
/* SHOW THE WARNING HERE */
} else {
/* HIDE THE WARNING HERE */
}
The warning can be implemented in many different ways: highlight the password field that the user is typing into, show a tooltip, display text next to the field. For simplicity, this code shows the warning as a previously created image, with appropriate alt text. Showing the warning means creating a new
tag with DOM scripting, dropping it into the page, and positioning it so that it’s next to the appropriate field. The image looks like this:
You know the position of the field the user is typing into (from its offsetTop and offsetLeft properties) and how wide it is (from its offsetWidth properties), so use createElement to make the new img element, and then absolutely position it with style properties so that it appears in the appropriate place (near to the text field).
The image is a transparent PNG with an alpha channel, so that the drop shadow appears nicely over whatever else is on the page. Because Internet Explorer version 6 and below doesn’t handle transparent PNGs correctly, you need to use the AlphaImageLoader technique to make the image appear correctly.
newimage = document.createElement('img');
newimage.src = ""http://farm3.static.flickr.com/2145/2067574980_3ddd405905_o_d.png"";
newimage.style.position = ""absolute"";
newimage.style.top = (targ.offsetTop - 73) + ""px"";
newimage.style.left = (targ.offsetLeft + targ.offsetWidth - 5) + ""px"";
newimage.style.zIndex = ""999"";
newimage.setAttribute(""alt"", ""Warning: Caps Lock is on"");
if (newimage.runtimeStyle) {
// PNG transparency for IE
newimage.runtimeStyle.filter += ""progid:DXImageTransform.Microsoft.AlphaImageLoader(src='http://farm3.static.flickr.com/2145/2067574980_3ddd405905_o_d.png',sizingMethod='scale')"";
}
document.body.appendChild(newimage);
Note that the alt text on the image is also correctly set. Next, all these parts need to be pulled together. On page load, identify all the password fields on the page, and attach a keypress handler to each. (This only needs to be done for password fields because the user can see if Caps Lock is on in ordinary text fields.)
var inps = document.getElementsByTagName(""input"");
for (var i=0, l=inps.length; i
The “create an image” code from above should only be run if the image is not already showing, so instead of creating a newimage object, create the image and attach it to the password field so that it can be checked for later (and not shown if it’s already showing). For safety, all the code should be wrapped up in its own object, so that its functions don’t collide with anyone else’s functions. So, create a single object called capslock and make all the functions be named methods of the object:
var capslock = {
...
keypress: function(e) {
}
...
}
Also, the “create an image” code is saved into its own named function, show_warning(), and the converse “remove the image” code into hide_warning(). This has the advantage that developers can include the JavaScript library that has been written here, but override what actually happens with their own code, using something like:
And that’s all. Simply include the JavaScript library in your pages, override what happens on a warning if that’s more appropriate for what you’re doing, and that’s all you need.
See the script in action.",2007,Stuart Langridge,stuartlangridge,2007-12-04T00:00:00+00:00,https://24ways.org/2007/capturing-caps-lock/,code
88,"Think First, Code Later","This is a story that’s best told from the end, and it’s probably one you’re all familiar with.
You, or someone just like you, have been building a website, probably as part of a skilled and capable team. You’re a front-end developer, focusing on JavaScript – it’s either your sole responsibility or shared around. It’s quite a big job, been going on for months, and at last it feels like you’re reaching the end of it.
But, in a brief moment of downtime, you step back and take a look at the code as a whole. You notice that the folder called “jQuery plugins” suddenly looks rather full, and maybe there’s evidence of several methods of doing the same thing; there are loads of little niggly fixes in the bug tracker; and every place you use Ajax the structure of the data is slightly different. You sigh, and your shoulders droop slightly, and you think “Yeah, we’ll do that more cleanly next time.”
The thing is, you probably already know how to rewrite the start of this story to make the ending work better. This situation is not really anyone’s fault – it’s just an accumulation of all the things you decided along the way, all the things you agreed you’d fix later that have disappeared into the black hole of technical debt, and accomodating all the “can we just…?” requests from around the team and the client.
So, the solution to this is easy, right? More interminable planning meetings, more tightly controlled and documented specifications, less freedom to innovate, to try out new ideas and enjoy what you’re doing.
Wait, that sounds even less fun than the old way.
Minimum viable planning
Actually, planning and specifications are exactly what you need, but the way you go about them can make a real difference, both to the quality of your code, and the quality of your life as a developer. It can be as simple as being a little more thoughtful before starting on any new piece of functionality. Involve your whole team if possible, or at least those working on what you’re doing. Canvass opinions and work out what the solution to the problem might look like first, rather than coding speculatively to find out.
There are easy ways you can get into this habit of putting the thought and design up front, and it doesn’t have to mean spending more time on the project as a whole. It also doesn’t have to result in reams of functional specifications. Instead, let the code itself form the specification.
As JavaScript applications become more complex, unit testing is becoming ever more important. So embrace it, whether you prefer QUnit, or Mocha, or any of the other JavaScript testing frameworks out there. The TDD (or test-driven development) pattern is all about writing the tests first and then writing functional code to pass those tests; or, if you prefer, code that meets the specification given by the tests.
Sounds like a hassle at first, but once you get into the rhythm of it you should find that the time spent writing tests up front is no greater, and often significantly less, than the time you would have spent fixing bugs afterwards.
If what you’re working on requires an API between client and server (usually Ajax but this can apply to any method of sending or receiving data) then spend a bit of time with the back-end developer to design the data contracts, before either of you cut any code. Work out what the API endpoints are going to be, and what the data structure you’ll get back from a certain endpoint looks like. A mock JSON object documented on a wiki is enough and it can be atomic. Don’t worry about planning the entire project at once, just plan enough to get on with your current tasks.
Definition in this way doesn’t have to make your API immutable – change is still fine – but if you know roughly where you’re heading, then not only can your team’s efforts become more parallel, but you’re far more likely to have an easier time making it all work. And again, you have a specification – the shape of the data – to write your JavaScript against.
Putting everything together, you end up with a logical flow of development, from the specification agreed with the client (your backlog), to the specification agreed with your team (the API contract design), to the specification agreed with your code (your unit tests). Hopefully, there will be ample clues in all of this to inform your front-end library choices, because by then you should have a better picture of what you’re going to need.
What the framework?
As a JavaScript developer predominantly, these are the choices I’m particularly interested in – how and why you use JavaScript libraries and frameworks, both what you expect from them and what you actually get.
If we look back at how web development, and specifically JavaScript development has progressed – from the earliest days of using lines and lines of Dreamweaver code-barf to make an image rollover effect, to today’s large frameworks that handle working with the DOM, Ajax communication and visual effects all in one hit – the purpose of it is clear: to smooth over the inconsistent bumps between browsers and give a solid, reliable, predictable base on which to put our desired functionality.
Understanding what we expect the language as a specification to do, and matching that to what we observe browsers actually doing, and then smoothing out the differences, is a big job. Since the language and the implementations are also changing as we go along, it also feels like a never-ending job. So make full use of this valuable effort. Use jQuery or YUI or anything else you’re comfortable with, but it still pays to think early on about what you need your library to do and what the best choice is to meet that need.
I’ve come in to projects as a fixer and found, to take a recent example, that jQuery UI was being used just to provide a date picker and a modal effect. That’s a lot of code weight to provide two fairly simple pieces of functionality that could easily be covered by smaller plugins. Which isn’t to say that jQuery UI itself is a bad choice, but I could see that it had been included late on just to do those things, whereas a more considered approach would have been to put the library in early and use it more universally.
There are other choices, too. If you automatically throw in jQuery (or whatever your favourite main library is) to a small site with limited functionality, you might only touch a tiny fraction of its scope. In my own development I started looking at what I actually needed from a JavaScript library. For a simple project like What the Framework?, all jQuery needed to do was listen for .ready() and then perform some light DOM selection before handing over to a client-side MVC framework. So perhaps there was another way to go about this while still avoiding the cross-browser headaches.
Deleting jQuery
But the jQuery pattern is compelling and familiar. And once you’re comfortable with something, it’s a bit of an effort to force yourself out of that comfort zone and learn. But looking back at my whole career, I realised that I’ve relearned pretty much everything I do, probably several times, since I started out. So it’s worth keeping in mind that learning and trying new things is how development has advanced to where it is now, and how it will keep advancing in the future.
In the end this lead me to Ender, which is billed as an NPM-style package manager for the browser, letting you search for and manage small, loosely coupled modules and their dependencies, and compile them to one file with a common API.
For What the Framework I ended up with a set of DOM tools, Underscore and Knockout, all minified into 25kb of JavaScript. This compares really well with 32kb minified for jQuery on its own, and Ender’s use of the dollar variable and the jQuery-like syntax in many modules makes switching over a low-friction experience.
On more complex projects, where you’re really going to use all the features of something like jQuery, but want to minimise the loading of other dependencies when you don’t need them, I’ve recently started looking at Jam. This uses the RequireJS pattern to compile commonly used code into a library file and then manage dependencies and bring in others on a per-page basis depending on how you need it. Again, it all comes down to thinking about what you need and using it only when you need it. And the configurability of tools like Ender or Jam allow you to be responsive to changing requirements as your project grows.
There is no right answer
That’s not to say this way of working automatically makes things easier. It doesn’t. On a large, long-running project or one where future functionality is unknown, it’s still hard to predict and plan for everything – at least until crystal balls as a service come about. But by including strong engineering practices in your front-end, and trying to minimise technical debt, you’re at least giving yourself a decent safety net to guard against the “can we just…?” tendencies that are a fact of life.
So, really, this is not an advocation of using a particular technology or framework, because I can’t tell you what works for you or your team. But what I can tell you is that working this way round has done wonders for my productivity and enthusiasm, both for code quality and for trying out new libraries. Give it a go, you might like it!",2012,Stephen Fulljames,stephenfulljames,2012-12-07T00:00:00+00:00,https://24ways.org/2012/think-first-code-later/,process
201,Lint the Web Forward With Sonarwhal,"Years ago, when I was in a senior in college, much of my web development courses focused on two things: the basics like HTML and CSS (and boy, do I mean basic), and Adobe Flash. I spent many nights writing ActionScript 3.0 to build interactions for the websites that I would add to my portfolio. A few months after graduating, I built one website in Flash for a client, then never again. Flash was dying, and it became obsolete in my résumé and portfolio.
That was my first lesson in the speed at which things change in technology, and what a daunting realization that was as a new graduate looking to enter the professional world. Now, seven years later, I work on the Microsoft Edge team where I help design and build a tool that would have lessened my early career anxieties: sonarwhal.
Sonarwhal is a linting tool, built by and for the web community. The code is open source and lives under the JS Foundation. It helps web developers and designers like me keep up with the constant change in technology while simultaneously teaching how to code better websites.
Introducing sonarwhal’s mascot Nellie
Good web development is hard. It is more than HTML, CSS, and JavaScript: developers are expected to have a grasp of accessibility, performance, security, emerging standards, and more, all while refreshing this knowledge every few months as the web evolves. It’s a lot to keep track of.
Web development is hard
Staying up-to-date on all this knowledge is one of the driving forces for developing this scanning tool. Whether you are just starting out, are a student, or you have over a decade of experience, the sonarwhal team wants to help you build better websites for all browsers.
Currently sonarwhal checks for best practices in five categories: Accessibility, Interoperability, Performance, PWAs, and Security. Each check is called a “rule”. You can configure them and even create your own rules if you need to follow some specific guidelines for your project (e.g. validate analytics attributes, title format of pages, etc.).
You can use sonarwhal in two ways:
An online version, that provides a quick and easy way to scan any public website.
A command line tool, if you want more control over the configuration, or want to integrate it into your development flow.
The Online Scanner
The online version offers a streamlined way to scan a website; just enter a URL and you will get a web page of scan results with a permalink that you can share and revisit at any time.
The online version of sonarwal
When my team works on a new rule, we spend the bulk of our time carefully researching each subject, finding sources, and documenting it rather than writing the rule’s code. Not only is it important that we get you the right results, but we also want you to understand why something is failing. Next to each failing rule you’ll find a link to its detailed documentation, explaining why you should care about it, what exactly we are testing, examples that pass and examples that don’t, and useful links to even more in-depth documentation if you are interested in the subject.
We hope that between reading the documentation and continued use of sonarwhal, developers can stay on top of best practices. As devs continue to build sites and identify recurring issues that appear in their results, they will hopefully start to automatically include those missing elements or fix those pieces of code that are producing errors. This also isn’t a one-way communication: the documentation is not only available on the sonarwhal site, but also on GitHub for editing so you can help us make it even better!
A results report
The current configuration for the online scanner is very strict, so it might hurt your feelings (it did when I first tested it on my personal website). But you can configure sonarwhal to any level of strictness as well as customize the command line tool to your needs!
Sonarwhal’s CLI
The CLI gives you full control of sonarwhal: what rules to use, tweaks to them, domains that are out of your control, and so on. You will need the latest node LTS (v8) or Stable (v9) and your favorite package manager, such as npm:
npm install -g sonarwhal
You can now run sonarwhal from anywhere via:
sonarwhal https://example.com
Using the CLI
The configuration is done via a .sonarwhalrc file. When analyzing a site, if no file is available, you will be prompted to answer a series of questions:
What connector do you want to use? Connectors are what sonarwhal uses to access a website and gather all the information about the requests, resources, HTML, etc. Currently it supports jsdom, Microsoft Edge, and Google Chrome.
What formatter? This is how you want to see the results: summary, stylish, etc. Make sure to look at the full list. Some are concise for, perfect for a quick build assessment, while others are more verbose and informative.
Do you want to use the recommended rules configuration? Rules are the things we are validating. Unless you’ve read the documentation and know what you are doing, first timers should probably use the recommended configuration.
What browsers are you targeting? One of the best features of sonarwhal is that rules can adapt their feedback depending on your targeted browsers, suggesting to add or remove things. For example, the rule “Highest Document Mode” will tell you to add the “X-UA-Compatible” header if IE10 or lower is targeted or remove if the opposite is true.
sonarwhal configuration generator questions
Once you answer all these questions the scan will start and you will have a .sonarwhalrc file similar to the following:
{
""connector"": {
""name"": ""jsdom"",
""options"": {
""waitFor"": 1000
}
},
""formatters"": ""stylish"",
""rulesTimeout"": 120000,
""rules"": {
""apple-touch-icons"": ""error"",
""axe"": ""error"",
""content-type"": ""error"",
""disown-opener"": ""error"",
""highest-available-document-mode"": ""error"",
""validate-set-cookie-header"": ""warning"",
// ...
}
}
You should see the scan initiate in the command line and within a few seconds the results should start to appear. Remember, the scan results will look different depending on which formatter you selected so try each one out to see which one you like best.
sonarwhal results on my website and hurting my feelings 💔
Now that you have a list of errors, you can get to work improving the site! Note though, that when you scan your website, it scans all the resources on that page and if you’ve added something like analytics or fonts hosted elsewhere, you are unable to change those files. You can configure the CLI to ignore files from certain domains so that you are only getting results for files you are in control of.
The documentation should give enough guidance on how to fix the errors, but if it’s insufficient, please help us and suggest edits or contribute back to it. This is a community effort and chances are someone else will have the same question as you.
When I scanned both my websites, sonarwhal alerted me to not having an Apple Touch Icon. If I search on the web as opposed to using the sonarwhal documentation, the first top 3 results give me outdated information: I need to include many different icon sizes. I don’t need to include all the different size icons that target different devices. Declaring one icon sized 180px x 180px will provide a large enough icon for devices and it will scale down as appropriate for people on older devices.
The information at the top of the search results isn’t always the correct answer to an issue and we don’t want you to have to search through outdated documentation. As sonarwhal’s capabilities expand, the goal is for it to be the one stop shop for helping preflight your website.
The journey up until now and looking forward
On the Microsoft Edge team, we’re passionate about empowering developers to build great websites. Every day we see so many sites come through our issue tracker. (Thanks for filing those bugs, they help us make Microsoft Edge better and better!) Some issues we see over and over are honest mistakes or outdated ‘best practices’ that could be avoided, so we built this tool to help everyone help make the web a better place.
When we decided to create sonarwhal, we wanted to create a tool that would help developers write better and more up-to-date code for their websites. We want sonarwhal to be useful to anyone so, early on, we defined three guiding principles we’ve used along the way:
Community Driven. We build for the community’s best interests. The web belongs to everyone and this project should too. Not only is it open source, we’ve also donated it to the JS Foundation and have an inclusive governance model that welcomes the collaboration of anyone, individual or company.
User Centric. We want to put the user at the center, making sonarwhal configurable for your needs and easy to use no matter what your skill level is.
Collaborative. We didn’t want to reinvent the wheel, so we collaborated with existing tools and services that help developers build for the web. Some examples are aXe, snyk.io, Cloudinary, etc.
This is just the beginning and we still have lots to do. We’re hard at work on a backlog of exciting features for future releases, such as:
New rules for a variety of areas like performance, accessibility, security, progressive web apps, and more.
A plug-in for Visual Studio Code: we want sonarwhal to help you write better websites, and what better moment than when you are in your editor.
Configuration options for the online service: as we fine tune the infrastructure, the rule configuration for our scanner is locked, but we look forward to adding CLI customization options here in the near future.
This is a tool for the web community by the web community so if you are excited about sonarwhal, making a better web, and want to contribute, we have a few issues where you might be able to help. Also, don’t forget to check the rest of the sonarwhal GitHub organization. PRs are always welcome and appreciated!
Let us know what you think about the scanner at @NarwhalNellie on Twitter and we hope you’ll help us lint the web forward!",2017,Stephanie Drescher,stephaniedrescher,2017-12-02T00:00:00+00:00,https://24ways.org/2017/lint-the-web-forward-with-sonarwhal/,code
203,Jobs-to-Be-Done in Your UX Toolbox,"Part 1: What is JTBD?
The concept of a “job” in “Jobs-To-Be-Done” is neatly encapsulated by a oft-quoted line from Theodore Levitt:
“People want a quarter-inch hole, not a quarter inch drill”.
Even so, Don Norman pointed out that perhaps Levitt “stopped too soon” at what the real customer goal might be. In the “The Design of Everyday Things”, he wrote:
“Levitt’s example of the drill implying that the goal is really a hole is only partially correct, however. When people go to a store to buy a drill, that is not their real goal. But why would anyone want a quarter-inch hole? Clearly that is an intermediate goal. Perhaps they wanted to hang shelves on the wall. Levitt stopped too soon. Once you realize that they don’t really want the drill, you realize that perhaps they don’t really want the hole, either: they want to install their bookshelves. Why not develop methods that don’t require holes? Or perhaps books that don’t require bookshelves.”
In other words, a “job” in JTBD lingo is a way to express a user need or provide a customer-centric problem frame that’s independent of a solution. As Tony Ulwick says:
“A job is stable, it doesn’t change over time.”
An example of a job is “tiding you over from breakfast to lunch.” You could hire a donut, a flapjack or a banana for that mid-morning snack—whatever does the job. If you can arrive at a clearly identified primary job (and likely some secondary ones too), you can be more creative in how you come up with an effective solution while keeping the customer problem in focus.
The team at Intercom wrote a book on their application of JTBD. In it, Des Traynor cleverly characterised how JTBD provides a different way to think about solutions that compete for the same job:
“Economy travel and business travel are both capable candidates applying for [the job: Get me face-to-face with my colleague in San Francisco], though they’re looking for significantly different salaries. Video conferencing isn’t as capable, but is willing to work for a far smaller salary. I’ve a hiring choice to make.”
So far so good: it’s relatively simple to understand what a job is, once you understand how it’s different from a “task”. Business consultant and Harvard professor Clay Christensen talks about the concept of “hiring” a product to do a “job”, and firing it when something better comes along. If you’re a company that focuses solutions on the customer job, you’re more likely to succeed. You’ll find these concepts often referred to as “Jobs-to-be-Done theory”. But the application of Jobs-to-Be-Done theory is a little more complicated; it comprises several related approaches.
I particularly like Jim Kalbach’s description of how JTBD is a “lens through which to understand value creation”. But it is also more. In my view, it’s a family of frameworks and methods—and perhaps even a philosophy.
Different facets in a family of frameworks
JTBD has its roots in market research and business strategy, and so it comes to the research table from a slightly different place compared to traditional UX or design research—we have our roots in human-computer interaction and ergonomics. I’ve found it helpful to keep in mind is that the application of JTBD theory is an evolving beast, so it’s common to find contradictions across different resources. My own use of it has varied from project to project. In speaking to others who have adopted it in different measures, it seems that we have all applied it in somewhat multifarious ways. As we like to often say in interviews: there are no wrong answers.
Outcome Driven Innovation
Tony Ulwick’s version of the JTBD history began with Outcome Driven Innovation (ODI), and this approach is best outlined in his seminal article published in the Harvard Business Review in 2002. To understand his more current JTBD approach in his new book “Jobs to Be Done: Theory to Practice”, I actually found it beneficial to read his approach in the original 2002 article for a clearer reference point.
In the earlier article, Ulwick presented a rigorous approach that combines interviews, surveys and an “opportunity” algorithm—a sequence of steps to determine the business opportunity. ODI centres around working with “desired outcome statements” that you unearth through interviews, followed by a means to quantify the gap between importance and satisfaction in a survey to different types of customers.
Since 2008, Ulwick has written about using job maps to make sense of what the customer may be trying to achieve. In a recent article, he describes the aim of the activity is “to discover what the customer is trying to get done at different points in executing a job and what must happen at each juncture in order for the job to be carried out successfully.”
A job map is not strictly a journey map, however tempting it is to see it that way. From a UX perspective, is one of many models we can use—and as our research team at Clearleft have found, how we use model can depend on the nature of the jobs we’ve uncovered in interviews and the characteristics of the problem we’re attempting to solve.
Figure 1. Universal job map
Ulwick’s current methodology is outlined in his new book, where he describes a complete end-to-end process: from customer and competitor research to framing market and product strategy.
The Jobs-To-Be-Done Interview
Back in 2013, I attended a workshop by Chris Spiek and Bob Moesta from the ReWired Group on JTBD at the behest of a then-MailChimp colleague, and I came away excited about their approach to product research. It felt different from anything I’d done before and for the first time in years, I felt that I was genuinely adding something new to my research toolbox.
A key idea is that if you focus on the stories of those who switched to you, and those who switch away from you, you can uncover the core jobs through looking at these opposite ends of engagement.
This framework centres around the JTBD interview method, which harnesses the power of a narrative framework to elicit the real reasons why someone “hired” something to do a job—be it something physical like a new coffee maker, or a digital service, such as a to-do list app. As you interview, you are trying to unearth the context around the key moments on the JTBD timeline (Figure 2). A common approach is to begin from the point the customer might have purchased something, back to the point where the thought of buying this thing first occurred to them.
Figure 2. JTBD Timeline
Figure 3. The Four Forces
The Forces Diagram (Figure 3) is a post-interview analysis tool where you can map out what causes customers to switch to something new and what holds them back.
The JTBD interview is effective at identifying core and secondary jobs, as well as some context around the user need. Because this method is designed to extract the story from the interviewee, it’s a powerful way to facilitate recall. Having done many such interviews, I’ve noticed one interesting side effect: participants often remember more details later on after the conversation has formally ended. It is worth scheduling a follow-up phone call or keep the channels open.
Strengths aside, it’s good to keep in mind that the JTBD interview is still primarily an interview technique, so you are relying on the context from the interviewee’s self-reported perspective. For example, a stronger research methodology combines JTBD interviews with contextual research and quantitative methods.
Job Stories
Alan Klement is credited for coming up with the term “job story” to describe the framing of jobs for product design by the team at Intercom:
“When … I want to … so I can ….”
Figure 4. Anatomy of a Job Story
Unlike a user story that traditionally frames a requirement around personas, job stories frame the user need based on the situation and context. Paul Adams, the VP of Product at Intercom, wrote:
“We frame every design problem in a Job, focusing on the triggering event or situation, the motivation and goal, and the intended outcome. […] We can map this Job to the mission and prioritise it appropriately. It ensures that we are constantly thinking about all four layers of design. We can see what components in our system are part of this Job and the necessary relationships and interactions required to facilitate it. We can design from the top down, moving through outcome, system, interactions, before getting to visual design.”
Systems of Progress
Apart from advocating using job stories, Klement believes that a core tenet of applying JTBD revolves around our desire for “self-betterment”—and that focusing on everyone’s desire for self-betterment is core to a successful strategy.
In his book, Klement takes JTBD further to being a tool for change through applying systems thinking. There, he introduces the systems of progress and how it can help focus product strategy approach to be more innovative.
Coincidentally, I applied similar thinking on mapping systemic change when we were looking to improve users’ trust with a local government forum earlier this year. It’s not just about capturing and satisfying the immediate job-to-be-done, it’s about framing the job so that you can a clear vision forward on how you can help your users improve their lives in the ways they want to.
This is really the point where JTBD becomes a philosophy of practice.
Part 2: Mixing It Up
There has been some misunderstanding about how adopting JTBD means ditching personas or some of our existing design tools or research techniques. This couldn’t have been more wrong.
Figure 5: Jim Kalbach’s JTBD model
Jim Kalbach has used Outcome-Driven Innovation for around 10 years. In a 2016 article, he presents a synthesised model of how to think about that has key elements from ODI, Christensen’s theories and the structure of the job story.
More interestingly, Kalbach has also combined the use of mental models with JTBD.
Claire Menke of UDemy has written a comprehensive article about using personas, JTBD and customer journey maps together in order to communicate more complete story from the users’ perspective. Claire highlights an especially interesting point in her article as she described her challenges:
“After much trial and error, I arrived at a foundational research framework to suit every team’s needs — allowing everyone to share the same holistic understanding, but extract the type of information most applicable to their work.”
In other words, the organisational context you are in likely can dictate what works best—after all the goal is to arrive at the best user experience for your audiences. Intercom can afford to go full-on on applying JTBD theory as a dominant approach because they are a start-up, but a large company or organisation with multiple business units may require a mix of tools, outputs and outcomes.
JTBD is an immensely powerful approach on many fronts—you’ll find many different references that lists the ways you can apply JTBD. However, in the context of this discussion, it might also be useful to we examine where it lies in our models of how we think about our UX and product processes.
JTBD in the UX ecosystem
Figure 6. The Elements of User Experience (source)
There are many ways we have tried to explain the UX discipline but I think Jesse James Garrett’s Elements of User Experience is a good place to begin.
I sometimes also use little diagram to help me describe the different levels you might work at when you work through the complexity of designing and developing a product. A holistic UX strategy needs to address all the different levels for a comprehensive experience: your individual product UI, product features, product propositions and brand need to have a cohesive definition.
Figure 7. Which level of product focus?
We could, of course, also think about where it fits best within the double diamond.
Again, bearing in mind that JTBD has its roots in business strategy and market research, it is excellent at clarifying user needs, defining high-level specifications and content requirements. It is excellent for validating brand perception and value proposition —all the way down to your feature set. In other words, it can be extremely powerful all the way through to halfway of the second diamond. You could quite readily combine the different JTBD approaches because they have differences as much as overlaps. However, JTBD generally starts getting a little difficult to apply once we get to the details of UI design.
The clue lies in JTBD’s raison d’être: a job statement is solution independent. Hence, once we get to designing solutions, we potentially fall into a existential black hole.
That said, Jim Kalbach has a quick case study on applying JTBD to content design tucked inside the main article on a synthesised JTBD model. Alan Klement has a great example of how you could use UI to resolve job stories. You’ll notice that the available language of “jobs” drops off at around that point.
Job statements and outcome statements provide excellent “mini north-stars” as customer-oriented focal points, but purely satisfying these statements would not necessarily guarantee that you have created a seamless and painless user experience.
Playing well with others
You will find that JTBD plays well with Lean, and other strategy tools like the Value Proposition Canvas. With every new project, there is potential to harness the power of JTBD alongside our established toolbox.
When we need to understand complex contexts where cultural or socioeconomic considerations have to be taken into account, we are better placed with combining JTBD with more anthropological approaches. And while we might be able to evaluate if our product, website or app satisfies the customer jobs through interviews or surveys, without good old-fashioned usability testing we are unlikely to be able to truly validate why the job isn’t being represented as it should. In this case, individual jobs solved on the UI can be set up as hypotheses to be proven right or wrong.
The application of Jobs-to-be-Done is still evolving. I’ve found it to be very powerful and I struggle to remember what my UX professional life was like before I encountered it—it has completely changed my approach to research and design.
The fact JTBD is still evolving as a practice means we need to be watchful of dogma—there’s no right way to get a UX job done after all, it nearly always depends. At the end of the day, isn’t it about having the right tool for the right job?",2017,Steph Troeth,stephtroeth,2017-12-04T00:00:00+00:00,https://24ways.org/2017/jobs-to-be-done-in-your-ux-toolbox/,ux
43,Content Production Planning,"While everyone agrees that getting the content of a website right is vital to its success, unless you’re lucky enough to have an experienced editor or content strategist on board, planning content production often seems to fall through the cracks. One reason is that, for most of the team, it feels like someone else’s problem. Not necessarily a specific person’s problem. Just someone else’s. It’s only when everyone starts urgently asking when the content is going to be ready, that it becomes clear the answer is, “Not as soon as we’d like it”.
The good news is that there are some quick and simple things you can do, even if you’re not the official content person on a project, to get everyone on the same content planning page.
Content production planning boils down to answering three deceptively simple questions:
What content do you need?
How much of it do you need?
Who’s going to make it?
Even if it’s not your job to come up with the answers, by asking these questions early enough and agreeing who is going to come up with the answers, you’ll be a long way towards avoiding the last-minute content problems which so often plague projects.
How much content do we need?
People tend to underestimate two crucial things about content: how much content they need, and how long that content takes to produce.
When I ask someone how big their website is – how many pages it contains – I usually double or triple the answer I get. That’s because almost everyone’s mental model of their website greatly underestimates its true size. You can see the problem for yourself if you look at a site map. Site maps are great at representing a mental model of a website. But because they’re a deliberate simplification they naturally lead us to underestimate how much content is involved in populating them.
Several years ago I was asked to help a client create a new microsite (their word) which they wanted ready in two weeks for a conference they were attending. Here’s the site map they had in mind. At first glance it looks like a pretty small website. Maybe twenty to thirty pages?
That’s what the client thought.
But see those boxes which are multiple boxes stacked on top of one another, for product categories, descriptions and supporting material? They’re known as page stacks, and page stacks are the content strategy equivalent of Here Be Dragons.
Say we have:
five product categories
each with five products
which all have two or three supporting documents
Those are still fairly small numbers. But small numbers multiplied by other small numbers tend to lead to big numbers.
5 categories = 5 category descriptions
plus
5 categories × 5 products each = 25 product descriptions
plus
25 products × 2.5 (average) supporting documents = 63 supporting documents
equals
93 pages
Suddenly our twenty- or thirty-page website is running towards one hundred.
That’s probably enough to get most project teams to sit up and take notice. But there’s still the danger of underestimating how long it’s going to take to create the content. After all, assuming the supporting documents already exist in some form, there are only about twenty-five to thirty pages of new copy to write.
How much work is it?
Again, we have the problem that small numbers when multiplied by other small numbers tend to lead to big numbers. Let’s make a rough guess that it’ll take four hours to write each product category and description page we need. That feels a little conservative if we’re writing stuff from scratch, but assuming the person doing it already knows the products fairly well it’s not unreasonable.
30 pages × 4 hours each = 120 hours
120 hours ÷ 7.5 working hours a day = 16 days
Ouch.
At this point it’s pretty clear we’re not getting this site launched in two weeks.
The goal is the conversation
By breaking down the site into its content components, and putting some rough estimates on how long each might take to produce, the client instantly realised that there was no way they would be ready to launch it in two weeks. Although we still didn’t know exactly when it would be ready, getting to that realisation right at the start of the project was a major win for everybody. Without it, the design agency would have bust a gut to get the design, front-end and CMS all done in double-quick time, only to find it was all for nothing as barely half the content was ready. As it was, an early discussion about content, albeit a brief one, bought everyone time to tackle the project properly, without pulling any long nights or working weekends.
If you haven’t been able to get people to discuss content plans for the project, these kinds of rough estimates should give you enough evidence to get everyone to start taking it seriously. Your goal is to get everyone on the project to a place where they are ready to talk in detail about who is going to create this content, and how long it’s really going to take them, and to get to those conversations before lack of content becomes a problem.
Be careful though. It’s best to talk in ranges and round numbers when your estimates are this uncertain. And watch those multipliers. Given small numbers multiplied by other small numbers lead to big numbers, changing just one number can greatly change the overall estimate. I like to run a couple of different scenarios to check what things look like if I’ve under- or overestimated either how many pages we’re going to need, or how long they’re going to take to create. For example:
Top end: 30 pages × 5 hours = 150 hours, or 20 days
Bottom end: 25 pages × 4 hours = 100 hours, or 13.3 days
So rather than say, “I estimate the content will take around sixteen days to produce”, I’m going to say, “I think the content will take about three to four weeks to produce”. Even with qualifiers like estimate and around, sixteen days sounds too precise. Whereas three to four weeks instantly conveys that this is just a rough figure.
Who’s going to make it?
So, people tend to underestimate two crucial things about content: how much content they need, and how long content takes to write. At this stage, you’re still in danger of the latter, because it’s tempting to simply estimate how much time content takes to write (or record, if we’re talking audio or visual content), and overlook all the other work that needs to goes on around it.
Take 24 ways as an example. In terms of our three deceptively simple questions: what is practical articles about web design; how many is twenty-four, one for each day of Advent; and who are experts working on the web, one to write each article.
But there’s another who you might not have considered.
Someone needs to select those authors in the first place, make sure they deliver their articles on time (and find someone to replace them if they don’t), review drafts, copy-edit and proofread final versions, upload them to the site, promote them, keep an eye on the comments and make sure there are still presents under the tree on Christmas morning.
Even if each of those tasks only takes an hour or so, it then needs multiplying by twenty-four (except the presents, obviously). And as we’ve already seen, small numbers multiplied by small numbers quickly turn into much bigger numbers. Just a few hours per article, when multiplied by twenty-four articles, easily multiplies up to days or even weeks of effort.
To get a more accurate estimate of how long the different kinds of content are going to take, you need to break down the content production work into its constituent stages, starting with planning, moving on through the main work of creation, to reviewing, approvals and finally publishing. You need to think about who needs to be involved at each step, and how much time they’ll need to do their bit.
Taken together, these things make up your content workflow. The workflow will be different for each organisation, but might look something like this:
Eddie the web editor will work out the key messages and objectives for each page, and agree them with Mo the marketing director.
Eddie will then get Cal, the copywriter, to write the first draft.
As part of that, Cal will interview Sam the subject expert to understand the intricacies of the subject and get all the facts straight.
Once Cal’s done the first draft, it’ll go to Sam to check for accuracy, while Eddie reviews it for style and message.
Once Cal has incorporated their feedback it’s time to get Mo to have a look at the final draft.
If Mo’s happy, it’ll get a final proofread, be uploaded to the CMS, and Mo will give the final sign-off and release it for publishing.
You can plot this on a table, with the stages of the content production process down the side, and the key roles or personnel along with top. Then the team can estimate how much time they think each of them needs at each stage.
Mo (marketing director)
Sam (subject expert)
Eddie (web editor)
Cal (copywriter)
Outline: define key messages and objectives
30 min
Review outline
15 min
First draft
30 min
3 hours
Review 1st draft
30 min
30 min
2nd draft
1 hour
Review 2nd draft
15 min
15 min
15 min
Final amendments
30 min
Proofread
15 min
Upload
15 min
Sign-off
10 min
TOTAL
40 min
1 hour 15 min
1 hour 30 min
4 hours 45 min
You can then bring out your calculator again, and come up with some more big scary numbers showing how much time it’s going to take for the whole team to get all the content needed not just written, but also planned, reviewed, approved and published.
With an experienced team you can run this exercise as a group workshop and get some fairly accurate estimates pretty quickly. If this is all a bit new to you, check out Gather Content’s Content Production Planning for Agencies ebook for a useful guide to common content roles, ballpark estimates for how much time each one needs on a typical piece of content, and how to run a process and estimating workshop to dig into them in more detail.
On a small team, one person might play many roles, but you should still sanity-check your estimates by breaking down the process and putting a rough estimate on each stage. With only a couple of people involved, it’s even easier to only include the core activity like writing or recording in your estimates, and forget to allow time for the planning, reviewing, proofreading, publishing and promoting you’ll still need to do. And even in a team of one, if at all possible you should find at least one other person to act as a second pair of eyes, and give anything you produce a quick once-over and proofread before it’s published.
Depending on the kind of content you’re making, you should also consider what will happen after it’s published. The full content life cycle should include promotion, monitoring and regular reviews to make sure content stays accurate and up to date. Making sure you have the time and resources available to do all those things for each piece of content is essential for creating a sustainable content programme.
The proof of the pudding
Even after digging into workflow and getting the whole team involved in estimating, you’re still largely in the realm of the guesstimate. The good news, though, is that you can quite quickly start finding out if your guesstimates are right or not. As soon as you can, pilot the production process with some real content. This is a double-win: you start finding out how long it really takes to produce all this fab new content, and you get real content to work with in designs and prototypes.
Once you’ve run a few things through your process, you’ll be able to refine your estimates, confirm your workflow, and give everyone involved a clear idea of when it will all be ready, and what you need from them.
Keeping it all on track
At this point I like to pull everything together into the content strategist’s favourite tool: the spreadsheet.
A simple content production checklist is a bit like a content inventory or audit, but for the content you don’t yet have, not the stuff already done. You can grab an example here.
Each piece of content gets its own row, with columns for basic information like page title, ID (which should match the site map), and who’s responsible for making it. You can capture simple details like target audience and key messages here too, though for more complex content, page description tables like those described by Relly Annett-Baker in “Extracting the Content” may be a better tool to use. Just adapt these columns to whatever makes sense for your content.
I then have columns to track where each piece is in the production process. I usually keep this simple, with a column each to mark whether it’s draft, final or uploaded. The status column on the left automatically shows the item’s status, using a simple traffic light colour scheme for whether the item is still to do (red), in draft (amber), or done (green). Seeing the whole thing slowly turn from red to green is a nice motivator.
If you want to track the workflow in more detail, a kanban board in a tool like Trello is a great way for a team to collaborate on content production, track each item’s progress, and keep an eye out for bottlenecks and delays.
Getting to the content strategy conversation
It’s a relatively simple exercise, then, to decide not just what kinds of pages you need, but also how many of them: put some rough estimates of effort on the tasks needed to create those pages – not just the writing, but all the other stages of planning, reviewing, approving, publishing and promoting – and then multiply all those things together. This will quickly bring some reality to grand visions and overambitious plans. Do it early enough, and even when the final big scary number is a lot bigger and scarier than everyone thought, you’ll still have time to do something about it.
As well as getting everyone on board for some proper content planning activities, that big scary number is your opportunity to get to the real core questions of content strategy: do we really need all this content? Where can existing content be reused and repurposed? How do we prioritise our efforts? What really matters to our readers and users?
Time and again, case studies show that less content delivers more: more leads, more sales, more self-service support and savings in the call centre. Although that argument is primarily one you should make from a good-for-the-users perspective, it doesn’t hurt to be able to make it from the cheaper-for-the-business perspective as well, and to have some big scary numbers to back that up.",2014,Sophie Dennis,sophiedennis,2014-12-17T00:00:00+00:00,https://24ways.org/2014/content-production-planning/,content
168,Unobtrusively Mapping Microformats with jQuery,"Microformats are everywhere. You can’t shake an electronic stick these days without accidentally poking a microformat-enabled site, and many developers use microformats as a matter of course. And why not? After all, why invent your own class names when you can re-use pre-defined ones that give your site extra functionality for free?
Nevertheless, while it’s good to know that users of tools such as Tails and Operator will derive added value from your shiny semantics, it’s nice to be able to reuse that effort in your own code.
We’re going to build a map of some of my favourite restaurants in Brighton. Fitting with the principles of unobtrusive JavaScript, we’ll start with a semantically marked up list of restaurants, then use JavaScript to add the map, look up the restaurant locations and plot them as markers.
We’ll be using a couple of powerful tools. The first is jQuery, a JavaScript library that is ideally suited for unobtrusive scripting. jQuery allows us to manipulate elements on the page based on their CSS selector, which makes it easy to extract information from microformats.
The second is Mapstraction, introduced here by Andrew Turner a few days ago. We’ll be using Google Maps in the background, but Mapstraction makes it easy to change to a different provider if we want to later.
Getting Started
We’ll start off with a simple collection of microformatted restaurant details, representing my seven favourite restaurants in Brighton. The full, unstyled list can be seen in restaurants-plain.html. Each restaurant listing looks like this:
12b Meeting House Lane
Brighton , UK
BN1 1HB
Telephone: +44 (0)1273 323 008
E-mail: info@riddleandfinns.co.uk
Since we’re dealing with a list of restaurants, each hCard is marked up inside a list item. Each restaurant is an organisation; we signify this by placing the classes fn and org on the element surrounding the restaurant’s name (according to the hCard spec, setting both fn and org to the same value signifies that the hCard represents an organisation rather than a person).
The address information itself is contained within a div of class adr. Note that the HTML
element is not suitable here for two reasons: firstly, it is intended to mark up contact details for the current document rather than generic addresses; secondly, address is an inline element and as such cannot contain the paragraphs elements used here for the address information.
A nice thing about microformats is that they provide us with automatic hooks for our styling. For the moment we’ll just tidy up the whitespace a bit; for more advanced style tips consult John Allsop’s guide from 24 ways 2006.
.vcard p {
margin: 0;
}
.adr {
margin-bottom: 0.5em;
}
To plot the restaurants on a map we’ll need latitude and longitude for each one. We can find this out from their address using geocoding. Most mapping APIs include support for geocoding, which means we can pass the API an address and get back a latitude/longitude point. Mapstraction provides an abstraction layer around these APIs which can be included using the following script tag:
While we’re at it, let’s pull in the other external scripts we’ll be using:
That’s everything set up: let’s write some JavaScript!
In jQuery, almost every operation starts with a call to the jQuery function. The function simulates method overloading to behave in different ways depending on the arguments passed to it. When writing unobtrusive JavaScript it’s important to set up code to execute when the page has loaded to the point that the DOM is available to be manipulated. To do this with jQuery, pass a callback function to the jQuery function itself:
jQuery(function() {
// This code will be executed when the DOM is ready
});
Initialising the map
The first thing we need to do is initialise our map. Mapstraction needs a div with an explicit width, height and ID to show it where to put the map. Our document doesn’t currently include this markup, but we can insert it with a single line of jQuery code:
jQuery(function() {
// First create a div to host the map
var themap = jQuery('
').css({
'width': '90%',
'height': '400px'
}).insertBefore('ul.restaurants');
});
While this is technically just a single line of JavaScript (with line-breaks added for readability) it’s actually doing quite a lot of work. Let’s break it down in to steps:
var themap = jQuery('
')
Here’s jQuery’s method overloading in action: if you pass it a string that starts with a < it assumes that you wish to create a new HTML element. This provides us with a handy shortcut for the more verbose DOM equivalent:
var themap = document.createElement('div');
themap.id = 'themap';
Next we want to apply some CSS rules to the element. jQuery supports chaining, which means we can continue to call methods on the object returned by jQuery or any of its methods:
var themap = jQuery('
').css({
'width': '90%',
'height': '400px'
})
Finally, we need to insert our new HTML element in to the page. jQuery provides a number of methods for element insertion, but in this case we want to position it directly before the we are using to contain our restaurants. jQuery’s insertBefore() method takes a CSS selector indicating an element already on the page and places the current jQuery selection directly before that element in the DOM.
var themap = jQuery('
').css({
'width': '90%',
'height': '400px'
}).insertBefore('ul.restaurants');
Finally, we need to initialise the map itself using Mapstraction. The Mapstraction constructor takes two arguments: the first is the ID of the element used to position the map; the second is the mapping provider to use (in this case google ):
// Initialise the map
var mapstraction = new Mapstraction('themap','google');
We want the map to appear centred on Brighton, so we’ll need to know the correct co-ordinates. We can use www.getlatlon.com to find both the co-ordinates and the initial map zoom level.
// Show map centred on Brighton
mapstraction.setCenterAndZoom(
new LatLonPoint(50.82423734980143, -0.14007568359375),
15 // Zoom level appropriate for Brighton city centre
);
We also want controls on the map to allow the user to zoom in and out and toggle between map and satellite view.
mapstraction.addControls({
zoom: 'large',
map_type: true
});
Adding the markers
It’s finally time to parse some microformats. Since we’re using hCard, the information we want is wrapped in elements with the class vcard. We can use jQuery’s CSS selector support to find them:
var vcards = jQuery('.vcard');
Now that we’ve found them, we need to create a marker for each one in turn. Rather than using a regular JavaScript for loop, we can instead use jQuery’s each() method to execute a function against each of the hCards.
jQuery('.vcard').each(function() {
// Do something with the hCard
});
Within the callback function, this is set to the current DOM element (in our case, the list item). If we want to call the magic jQuery methods on it we’ll need to wrap it in another call to jQuery:
jQuery('.vcard').each(function() {
var hcard = jQuery(this);
});
The Google maps geocoder seems to work best if you pass it the street address and a postcode. We can extract these using CSS selectors: this time, we’ll use jQuery’s find() method which searches within the current jQuery selection:
var streetaddress = hcard.find('.street-address').text();
var postcode = hcard.find('.postal-code').text();
The text() method extracts the text contents of the selected node, minus any HTML markup.
We’ve got the address; now we need to geocode it. Mapstraction’s geocoding API requires us to first construct a MapstractionGeocoder, then use the geocode() method to pass it an address. Here’s the code outline:
var geocoder = new MapstractionGeocoder(onComplete, 'google');
geocoder.geocode({'address': 'the address goes here');
The onComplete function is executed when the geocoding operation has been completed, and will be passed an object with the resulting point on the map. We just want to create a marker for the point:
var geocoder = new MapstractionGeocoder(function(result) {
var marker = new Marker(result.point);
mapstraction.addMarker(marker);
}, 'google');
For our purposes, joining the street address and postcode with a comma to create the address should suffice:
geocoder.geocode({'address': streetaddress + ', ' + postcode});
There’s one last step: when the marker is clicked, we want to display details of the restaurant. We can do this with an info bubble, which can be configured by passing in a string of HTML. We’ll construct that HTML using jQuery’s html() method on our hcard object, which extracts the HTML contained within that DOM node as a string.
var marker = new Marker(result.point);
marker.setInfoBubble(
'' + hcard.html() + '
'
);
mapstraction.addMarker(marker);
We’ve wrapped the bubble in a div with class bubble to make it easier to style. Google Maps can behave strangely if you don’t provide an explicit width for your info bubbles, so we’ll add that to our CSS now:
.bubble {
width: 300px;
}
That’s everything we need: let’s combine our code together:
jQuery(function() {
// First create a div to host the map
var themap = jQuery('
').css({
'width': '90%',
'height': '400px'
}).insertBefore('ul.restaurants');
// Now initialise the map
var mapstraction = new Mapstraction('themap','google');
mapstraction.addControls({
zoom: 'large',
map_type: true
});
// Show map centred on Brighton
mapstraction.setCenterAndZoom(
new LatLonPoint(50.82423734980143, -0.14007568359375),
15 // Zoom level appropriate for Brighton city centre
);
// Geocode each hcard and add a marker
jQuery('.vcard').each(function() {
var hcard = jQuery(this);
var streetaddress = hcard.find('.street-address').text();
var postcode = hcard.find('.postal-code').text();
var geocoder = new MapstractionGeocoder(function(result) {
var marker = new Marker(result.point);
marker.setInfoBubble(
'' + hcard.html() + '
'
);
mapstraction.addMarker(marker);
}, 'google');
geocoder.geocode({'address': streetaddress + ', ' + postcode});
});
});
Here’s the finished code.
There’s one last shortcut we can add: jQuery provides the $ symbol as an alias for jQuery. We could just go through our code and replace every call to jQuery() with a call to $(), but this would cause incompatibilities if we ever attempted to use our script on a page that also includes the Prototype library. A more robust approach is to start our code with the following:
jQuery(function($) {
// Within this function, $ now refers to jQuery
// ...
});
jQuery cleverly passes itself as the first argument to any function registered to the DOM ready event, which means we can assign a local $ variable shortcut without affecting the $ symbol in the global scope. This makes it easy to use jQuery with other libraries.
Limitations of Geocoding
You may have noticed a discrepancy creep in to the last example: whereas my original list included seven restaurants, the geocoding example only shows five. This is because the Google Maps geocoder incorporates a rate limit: more than five lookups in a second and it starts returning error messages instead of regular results.
In addition to this problem, geocoding itself is an inexact science: while UK postcodes generally get you down to the correct street, figuring out the exact point on the street from the provided address usually isn’t too accurate (although Google do a pretty good job).
Finally, there’s the performance overhead. We’re making five geocoding requests to Google for every page served, even though the restaurants themselves aren’t likely to change location any time soon. Surely there’s a better way of doing this?
Microformats to the rescue (again)! The geo microformat suggests simple classes for including latitude and longitude information in a page. We can add specific points for each restaurant using the following markup:
E-Kagen
22-23 Sydney Street
Brighton , UK
BN1 4EN
Telephone: +44 (0)1273 687 068
Lat/Lon:
50.827917 ,
-0.137764
As before, I used www.getlatlon.com to find the exact locations – I find satellite view is particularly useful for locating individual buildings.
Latitudes and longitudes are great for machines but not so useful for human beings. We could hide them entirely with display: none, but I prefer to merely de-emphasise them (someone might want them for their GPS unit):
.vcard .geo {
margin-top: 0.5em;
font-size: 0.85em;
color: #ccc;
}
It’s probably a good idea to hide them completely when they’re displayed inside an info bubble:
.bubble .geo {
display: none;
}
We can extract the co-ordinates in the same way we extracted the address. Since we’re no longer geocoding anything our code becomes a lot simpler:
$('.vcard').each(function() {
var hcard = $(this);
var latitude = hcard.find('.geo .latitude').text();
var longitude = hcard.find('.geo .longitude').text();
var marker = new Marker(new LatLonPoint(latitude, longitude));
marker.setInfoBubble(
'' + hcard.html() + '
'
);
mapstraction.addMarker(marker);
});
And here’s the finished geo example.
Further reading
We’ve only scratched the surface of what’s possible with microformats, jQuery (or just regular JavaScript) and a bit of imagination. If this example has piqued your interest, the following links should give you some more food for thought.
The hCard specification
Notes on parsing hCards
jQuery for JavaScript programmers – my extended tutorial on jQuery.
Dann Webb’s Sumo – a full JavaScript library for parsing microformats, based around some clever metaprogramming techniques.
Jeremy Keith’s Adactio Austin – the first place I saw using microformats to unobtrusively plot locations on a map. Makes clever use of hEvent as well.",2007,Simon Willison,simonwillison,2007-12-12T00:00:00+00:00,https://24ways.org/2007/unobtrusively-mapping-microformats-with-jquery/,code
249,Fast Autocomplete Search for Your Website,"Every website deserves a great search engine - but building a search engine can be a lot of work, and hosting it can quickly get expensive.
I’m going to build a search engine for 24 ways that’s fast enough to support autocomplete (a.k.a. typeahead) search queries and can be hosted for free. I’ll be using wget, Python, SQLite, Jupyter, sqlite-utils and my open source Datasette tool to build the API backend, and a few dozen lines of modern vanilla JavaScript to build the interface.
Try it out here, then read on to see how I built it.
First step: crawling the data
The first step in building a search engine is to grab a copy of the data that you plan to make searchable.
There are plenty of potential ways to do this: you might be able to pull it directly from a database, or extract it using an API. If you don’t have access to the raw data, you can imitate Google and write a crawler to extract the data that you need.
I’m going to do exactly that against 24 ways: I’ll build a simple crawler using wget, a command-line tool that features a powerful “recursive” mode that’s ideal for scraping websites.
We’ll start at the https://24ways.org/archives/ page, which links to an archived index for every year that 24 ways has been running.
Then we’ll tell wget to recursively crawl the website, using the --recursive flag.
We don’t want to fetch every single page on the site - we’re only interested in the actual articles. Luckily, 24 ways has nicely designed URLs, so we can tell wget that we only care about pages that start with one of the years it has been running, using the -I argument like this: -I /2005,/2006,/2007,/2008,/2009,/2010,/2011,/2012,/2013,/2014,/2015,/2016,/2017
We want to be polite, so let’s wait for 2 seconds between each request rather than hammering the site as fast as we can: --wait 2
The first time I ran this, I accidentally downloaded the comments pages as well. We don’t want those, so let’s exclude them from the crawl using -X ""/*/*/comments"".
Finally, it’s useful to be able to run the command multiple times without downloading pages that we have already fetched. We can use the --no-clobber option for this.
Tie all of those options together and we get this command:
wget --recursive --wait 2 --no-clobber
-I /2005,/2006,/2007,/2008,/2009,/2010,/2011,/2012,/2013,/2014,/2015,/2016,/2017
-X ""/*/*/comments""
https://24ways.org/archives/
If you leave this running for a few minutes, you’ll end up with a folder structure something like this:
$ find 24ways.org
24ways.org
24ways.org/2013
24ways.org/2013/why-bother-with-accessibility
24ways.org/2013/why-bother-with-accessibility/index.html
24ways.org/2013/levelling-up
24ways.org/2013/levelling-up/index.html
24ways.org/2013/project-hubs
24ways.org/2013/project-hubs/index.html
24ways.org/2013/credits-and-recognition
24ways.org/2013/credits-and-recognition/index.html
...
As a quick sanity check, let’s count the number of HTML pages we have retrieved:
$ find 24ways.org | grep index.html | wc -l
328
There’s one last step! We got everything up to 2017, but we need to fetch the articles for 2018 (so far) as well. They aren’t linked in the /archives/ yet so we need to point our crawler at the site’s front page instead:
wget --recursive --wait 2 --no-clobber
-I /2018
-X ""/*/*/comments""
https://24ways.org/
Thanks to --no-clobber, this is safe to run every day in December to pick up any new content.
We now have a folder on our computer containing an HTML file for every article that has ever been published on the site! Let’s use them to build ourselves a search index.
Building a search index using SQLite
There are many tools out there that can be used to build a search engine. You can use an open-source search server like Elasticsearch or Solr, a hosted option like Algolia or Amazon CloudSearch or you can tap into the built-in search features of relational databases like MySQL or PostgreSQL.
I’m going to use something that’s less commonly used for web applications but makes for a powerful and extremely inexpensive alternative: SQLite.
SQLite is the world’s most widely deployed database, even though many people have never even heard of it. That’s because it’s designed to be used as an embedded database: it’s commonly used by native mobile applications and even runs as part of the default set of apps on the Apple Watch!
SQLite has one major limitation: unlike databases like MySQL and PostgreSQL, it isn’t really designed to handle large numbers of concurrent writes. For this reason, most people avoid it for building web applications.
This doesn’t matter nearly so much if you are building a search engine for infrequently updated content - say one for a site that only publishes new content on 24 days every year.
It turns out SQLite has very powerful full-text search functionality built into the core database - the FTS5 extension.
I’ve been doing a lot of work with SQLite recently, and as part of that, I’ve been building a Python utility library to make building new SQLite databases as easy as possible, called sqlite-utils. It’s designed to be used within a Jupyter notebook - an enormously productive way of interacting with Python code that’s similar to the Observable notebooks Natalie described on 24 ways yesterday.
If you haven’t used Jupyter before, here’s the fastest way to get up and running with it - assuming you have Python 3 installed on your machine. We can use a Python virtual environment to ensure the software we are installing doesn’t clash with any other installed packages:
$ python3 -m venv ./jupyter-venv
$ ./jupyter-venv/bin/pip install jupyter
# ... lots of installer output
# Now lets install some extra packages we will need later
$ ./jupyter-venv/bin/pip install beautifulsoup4 sqlite-utils html5lib
# And start the notebook web application
$ ./jupyter-venv/bin/jupyter-notebook
# This will open your browser to Jupyter at http://localhost:8888/
You should now be in the Jupyter web application. Click New -> Python 3 to start a new notebook.
A neat thing about Jupyter notebooks is that if you publish them to GitHub (either in a regular repository or as a Gist), it will render them as HTML. This makes them a very powerful way to share annotated code. I’ve published the notebook I used to build the search index on my GitHub account.
Here’s the Python code I used to scrape the relevant data from the downloaded HTML files. Check out the notebook for a line-by-line explanation of what’s going on.
from pathlib import Path
from bs4 import BeautifulSoup as Soup
base = Path(""/Users/simonw/Dropbox/Development/24ways-search"")
articles = list(base.glob(""*/*/*/*.html""))
# articles is now a list of paths that look like this:
# PosixPath('...24ways-search/24ways.org/2013/why-bother-with-accessibility/index.html')
docs = []
for path in articles:
year = str(path.relative_to(base)).split(""/"")[1]
url = 'https://' + str(path.relative_to(base).parent) + '/'
soup = Soup(path.open().read(), ""html5lib"")
author = soup.select_one("".c-continue"")[""title""].split(
""More information about""
)[1].strip()
author_slug = soup.select_one("".c-continue"")[""href""].split(
""/authors/""
)[1].split(""/"")[0]
published = soup.select_one("".c-meta time"")[""datetime""]
contents = soup.select_one("".e-content"").text.strip()
title = soup.find(""title"").text.split("" ◆"")[0]
try:
topic = soup.select_one(
'.c-meta a[href^=""/topics/""]'
)[""href""].split(""/topics/"")[1].split(""/"")[0]
except TypeError:
topic = None
docs.append({
""title"": title,
""contents"": contents,
""year"": year,
""author"": author,
""author_slug"": author_slug,
""published"": published,
""url"": url,
""topic"": topic,
})
After running this code, I have a list of Python dictionaries representing each of the documents that I want to add to the index. The list looks something like this:
[
{
""title"": ""Why Bother with Accessibility?"",
""contents"": ""Web accessibility (known in other fields as inclus..."",
""year"": ""2013"",
""author"": ""Laura Kalbag"",
""author_slug"": ""laurakalbag"",
""published"": ""2013-12-10T00:00:00+00:00"",
""url"": ""https://24ways.org/2013/why-bother-with-accessibility/"",
""topic"": ""design""
},
{
""title"": ""Levelling Up"",
""contents"": ""Hello, 24 ways. Iu2019m Ashley and I sell property ins..."",
""year"": ""2013"",
""author"": ""Ashley Baxter"",
""author_slug"": ""ashleybaxter"",
""published"": ""2013-12-06T00:00:00+00:00"",
""url"": ""https://24ways.org/2013/levelling-up/"",
""topic"": ""business""
},
...
My sqlite-utils library has the ability to take a list of objects like this and automatically create a SQLite database table with the right schema to store the data. Here’s how to do that using this list of dictionaries.
import sqlite_utils
db = sqlite_utils.Database(""/tmp/24ways.db"")
db[""articles""].insert_all(docs)
That’s all there is to it! The library will create a new database and add a table to it called articles with the necessary columns, then insert all of the documents into that table.
(I put the database in /tmp/ for the moment - you can move it to a more sensible location later on.)
You can inspect the table using the sqlite3 command-line utility (which comes with OS X) like this:
$ sqlite3 /tmp/24ways.db
sqlite> .headers on
sqlite> .mode column
sqlite> select title, author, year from articles;
title author year
------------------------------ ------------ ----------
Why Bother with Accessibility? Laura Kalbag 2013
Levelling Up Ashley Baxte 2013
Project Hubs: A Home Base for Brad Frost 2013
Credits and Recognition Geri Coady 2013
Managing a Mind Christopher 2013
Run Ragged Mark Boulton 2013
Get Started With GitHub Pages Anna Debenha 2013
Coding Towards Accessibility Charlie Perr 2013
...
There’s one last step to take in our notebook. We know we want to use SQLite’s full-text search feature, and sqlite-utils has a simple convenience method for enabling it for a specified set of columns in a table. We want to be able to search by the title, author and contents fields, so we call the enable_fts() method like this:
db[""articles""].enable_fts([""title"", ""author"", ""contents""])
Introducing Datasette
Datasette is the open-source tool I’ve been building that makes it easy to both explore SQLite databases and publish them to the internet.
We’ve been exploring our new SQLite database using the sqlite3 command-line tool. Wouldn’t it be nice if we could use a more human-friendly interface for that?
If you don’t want to install Datasette right now, you can visit https://search-24ways.herokuapp.com/ to try it out against the 24 ways search index data. I’ll show you how to deploy Datasette to Heroku like this later in the article.
If you want to install Datasette locally, you can reuse the virtual environment we created to play with Jupyter:
./jupyter-venv/bin/pip install datasette
This will install Datasette in the ./jupyter-venv/bin/ folder. You can also install it system-wide using regular pip install datasette.
Now you can run Datasette against the 24ways.db file we created earlier like so:
./jupyter-venv/bin/datasette /tmp/24ways.db
This will start a local webserver running. Visit http://localhost:8001/ to start interacting with the Datasette web application.
If you want to try out Datasette without creating your own 24ways.db file you can download the one I created directly from https://search-24ways.herokuapp.com/24ways-ae60295.db
Publishing the database to the internet
One of the goals of the Datasette project is to make deploying data-backed APIs to the internet as easy as possible. Datasette has a built-in command for this, datasette publish. If you have an account with Heroku or Zeit Now, you can deploy a database to the internet with a single command. Here’s how I deployed https://search-24ways.herokuapp.com/ (running on Heroku’s free tier) using datasette publish:
$ ./jupyter-venv/bin/datasette publish heroku /tmp/24ways.db --name search-24ways
-----> Python app detected
-----> Installing requirements with pip
-----> Running post-compile hook
-----> Discovering process types
Procfile declares types -> web
-----> Compressing...
Done: 47.1M
-----> Launching...
Released v8
https://search-24ways.herokuapp.com/ deployed to Heroku
If you try this out, you’ll need to pick a different --name, since I’ve already taken search-24ways.
You can run this command as many times as you like to deploy updated versions of the underlying database.
Searching and faceting
Datasette can detect tables with SQLite full-text search configured, and will add a search box directly to the page. Take a look at http://search-24ways.herokuapp.com/24ways-b607e21/articles to see this in action.
SQLite search supports wildcards, so if you want autocomplete-style search where you don’t need to enter full words to start getting results you can add a * to the end of your search term. Here’s a search for access* which returns articles on accessibility:
http://search-24ways.herokuapp.com/24ways-ae60295/articles?_search=acces%2A
A neat feature of Datasette is the ability to calculate facets against your data. Here’s a page showing search results for svg with facet counts calculated against both the year and the topic columns:
http://search-24ways.herokuapp.com/24ways-ae60295/articles?_search=svg&_facet=year&_facet=topic
Every page visible via Datasette has a corresponding JSON API, which can be accessed using the JSON link on the page - or by adding a .json extension to the URL:
http://search-24ways.herokuapp.com/24ways-ae60295/articles.json?_search=acces%2A
Better search using custom SQL
The search results we get back from ../articles?_search=svg are OK, but the order they are returned in is not ideal - they’re actually being returned in the order they were inserted into the database! You can see why this is happening by clicking the View and edit SQL link on that search results page.
This exposes the underlying SQL query, which looks like this:
select rowid, * from articles where rowid in (
select rowid from articles_fts where articles_fts match :search
) order by rowid limit 101
We can do better than this by constructing a custom SQL query. Here’s the query we will use instead:
select
snippet(articles_fts, -1, 'b4de2a49c8', '8c94a2ed4b', '...', 100) as snippet,
articles_fts.rank, articles.title, articles.url, articles.author, articles.year
from articles
join articles_fts on articles.rowid = articles_fts.rowid
where articles_fts match :search || ""*""
order by rank limit 10;
You can try this query out directly - since Datasette opens the underling SQLite database in read-only mode and enforces a one second time limit on queries, it’s safe to allow users to provide arbitrary SQL select queries for Datasette to execute.
There’s a lot going on here! Let’s break the SQL down line-by-line:
select
snippet(articles_fts, -1, 'b4de2a49c8', '8c94a2ed4b', '...', 100) as snippet,
We’re using snippet(), a built-in SQLite function, to generate a snippet highlighting the words that matched the query. We use two unique strings that I made up to mark the beginning and end of each match - you’ll see why in the JavaScript later on.
articles_fts.rank, articles.title, articles.url, articles.author, articles.year
These are the other fields we need back - most of them are from the articles table but we retrieve the rank (representing the strength of the search match) from the magical articles_fts table.
from articles
join articles_fts on articles.rowid = articles_fts.rowid
articles is the table containing our data. articles_fts is a magic SQLite virtual table which implements full-text search - we need to join against it to be able to query it.
where articles_fts match :search || ""*""
order by rank limit 10;
:search || ""*"" takes the ?search= argument from the page querystring and adds a * to the end of it, giving us the wildcard search that we want for autocomplete. We then match that against the articles_fts table using the match operator. Finally, we order by rank so that the best matching results are returned at the top - and limit to the first 10 results.
How do we turn this into an API? As before, the secret is to add the .json extension. Datasette actually supports multiple shapes of JSON - we’re going to use ?_shape=array to get back a plain array of objects:
JSON API call to search for articles matching SVG
The HTML version of that page shows the time taken to execute the SQL in the footer. Hitting refresh a few times, I get response times between 2 and 5ms - easily fast enough to power a responsive autocomplete feature.
A simple JavaScript autocomplete search interface
I considered building this using React or Svelte or another of the myriad of JavaScript framework options available today, but then I remembered that vanilla JavaScript in 2018 is a very productive environment all on its own.
We need a few small utility functions: first, a classic debounce function adapted from this one by David Walsh:
function debounce(func, wait, immediate) {
let timeout;
return function() {
let context = this, args = arguments;
let later = () => {
timeout = null;
if (!immediate) func.apply(context, args);
};
let callNow = immediate && !timeout;
clearTimeout(timeout);
timeout = setTimeout(later, wait);
if (callNow) func.apply(context, args);
};
};
We’ll use this to only send fetch() requests a maximum of once every 100ms while the user is typing.
Since we’re rendering data that might include HTML tags (24 ways is a site about web development after all), we need an HTML escaping function. I’m amazed that browsers still don’t bundle a default one of these:
const htmlEscape = (s) => s.replace(
/>/g, '>'
).replace(
/Autocomplete search
And now the autocomplete implementation itself, as a glorious, messy stream-of-consciousness of JavaScript:
// Embed the SQL query in a multi-line backtick string:
const sql = `select
snippet(articles_fts, -1, 'b4de2a49c8', '8c94a2ed4b', '...', 100) as snippet,
articles_fts.rank, articles.title, articles.url, articles.author, articles.year
from articles
join articles_fts on articles.rowid = articles_fts.rowid
where articles_fts match :search || ""*""
order by rank limit 10`;
// Grab a reference to the
const searchbox = document.getElementById(""searchbox"");
// Used to avoid race-conditions:
let requestInFlight = null;
searchbox.onkeyup = debounce(() => {
const q = searchbox.value;
// Construct the API URL, using encodeURIComponent() for the parameters
const url = (
""https://search-24ways.herokuapp.com/24ways-866073b.json?sql="" +
encodeURIComponent(sql) +
`&search=${encodeURIComponent(q)}&_shape=array`
);
// Unique object used just for race-condition comparison
let currentRequest = {};
requestInFlight = currentRequest;
fetch(url).then(r => r.json()).then(d => {
if (requestInFlight !== currentRequest) {
// Avoid race conditions where a slow request returns
// after a faster one.
return;
}
let results = d.map(r => `
${htmlEscape(r.author)} - ${r.year}
${highlight(r.snippet)}
`).join("""");
document.getElementById(""results"").innerHTML = results;
});
}, 100); // debounce every 100ms
There’s just one more utility function, used to help construct the HTML results:
const highlight = (s) => htmlEscape(s).replace(
/b4de2a49c8/g, ''
).replace(
/8c94a2ed4b/g, ' '
);
This is what those unique strings passed to the snippet() function were for.
Avoiding race conditions in autocomplete
One trick in this code that you may not have seen before is the way race-conditions are handled. Any time you build an autocomplete feature, you have to consider the following case:
User types acces
Browser sends request A - querying documents matching acces*
User continues to type accessibility
Browser sends request B - querying documents matching accessibility*
Request B returns. It was fast, because there are fewer documents matching the full term
The results interface updates with the documents from request B, matching accessibility*
Request A returns results (this was the slower of the two requests)
The results interface updates with the documents from request A - results matching access*
This is a terrible user experience: the user saw their desired results for a brief second, and then had them snatched away and replaced with those results from earlier on.
Thankfully there’s an easy way to avoid this. I set up a variable in the outer scope called requestInFlight, initially set to null.
Any time I start a new fetch() request, I create a new currentRequest = {} object and assign it to the outer requestInFlight as well.
When the fetch() completes, I use requestInFlight !== currentRequest to sanity check that the currentRequest object is strictly identical to the one that was in flight. If a new request has been triggered since we started the current request we can detect that and avoid updating the results.
It’s not a lot of code, really
And that’s the whole thing! The code is pretty ugly, but when the entire implementation clocks in at fewer than 70 lines of JavaScript, I honestly don’t think it matters. You’re welcome to refactor it as much you like.
How good is this search implementation? I’ve been building search engines for a long time using a wide variety of technologies and I’m happy to report that using SQLite in this way is genuinely a really solid option. It scales happily up to hundreds of MBs (or even GBs) of data, and the fact that it’s based on SQL makes it easy and flexible to work with.
A surprisingly large number of desktop and mobile applications you use every day implement their search feature on top of SQLite.
More importantly though, I hope that this demonstrates that using Datasette for an API means you can build relatively sophisticated API-backed applications with very little backend programming effort. If you’re working with a small-to-medium amount of data that changes infrequently, you may not need a more expensive database. Datasette-powered applications easily fit within the free tier of both Heroku and Zeit Now.
For more of my writing on Datasette, check out the datasette tag on my blog. And if you do build something fun with it, please let me know on Twitter.",2018,Simon Willison,simonwillison,2018-12-19T00:00:00+00:00,https://24ways.org/2018/fast-autocomplete-search-for-your-website/,code
326,Don't be eval(),"JavaScript is an interpreted language, and like so many of its peers it includes the all powerful eval() function. eval() takes a string and executes it as if it were regular JavaScript code. It’s incredibly powerful and incredibly easy to abuse in ways that make your code slower and harder to maintain. As a general rule, if you’re using eval() there’s probably something wrong with your design.
Common mistakes
Here’s the classic misuse of eval(). You have a JavaScript object, foo, and you want to access a property on it – but you don’t know the name of the property until runtime. Here’s how NOT to do it:
var property = 'bar';
var value = eval('foo.' + property);
Yes it will work, but every time that piece of code runs JavaScript will have to kick back in to interpreter mode, slowing down your app. It’s also dirt ugly.
Here’s the right way of doing the above:
var property = 'bar';
var value = foo[property];
In JavaScript, square brackets act as an alternative to lookups using a dot. The only difference is that square bracket syntax expects a string.
Security issues
In any programming language you should be extremely cautious of executing code from an untrusted source. The same is true for JavaScript – you should be extremely cautious of running eval() against any code that may have been tampered with – for example, strings taken from the page query string. Executing untrusted code can leave you vulnerable to cross-site scripting attacks.
What’s it good for?
Some programmers say that eval() is B.A.D. – Broken As Designed – and should be removed from the language. However, there are some places in which it can dramatically simplify your code. A great example is for use with XMLHttpRequest, a component of the set of tools more popularly known as Ajax. XMLHttpRequest lets you make a call back to the server from JavaScript without refreshing the whole page. A simple way of using this is to have the server return JavaScript code which is then passed to eval(). Here is a simple function for doing exactly that – it takes the URL to some JavaScript code (or a server-side script that produces JavaScript) and loads and executes that code using XMLHttpRequest and eval().
function evalRequest(url) {
var xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange = function() {
if (xmlhttp.readyState==4 && xmlhttp.status==200) {
eval(xmlhttp.responseText);
}
}
xmlhttp.open(""GET"", url, true);
xmlhttp.send(null);
}
If you want this to work with Internet Explorer you’ll need to include this compatibility patch.",2005,Simon Willison,simonwillison,2005-12-07T00:00:00+00:00,https://24ways.org/2005/dont-be-eval/,code
230,The Articulate Web Designer of Tomorrow,"You could say that we design to communicate, and that we seek emotive responses. It sounds straightforward, and it can be, but leaving it to chance isn’t wise. Many wander into web design without formal training, and whilst that certainly isn’t essential, we owe it to ourselves to draw on wider influences, learn from the past, and think smarter.
What knowledge can we ourselves explore in order to become better designers? In addition, how can we take this knowledge, investigate it through our unique discipline, and in turn speak more eloquently about what we do on the web? Below, I outline a number of things that I personally believe all designers should be using and exploring collectively.
Taking stock
Where we’re at is good. Finding clarity through web standards, we’ve ended up quite modernist in our approach, pursuing function, elegance and reduction. However, we’re not great at articulating our own design processes and principles to outsiders. Equally, we rely heavily on our instincts when deciding if something is or isn’t good. That’s fine, but we can better understand why things are the way they are by looking a little deeper, thereby helping us articulate what goes on in our design brains to our peers, our clients and to normal humans.
As designers we use ideas, concepts, text and images. We apply our ideas and experience, imposing order and structure to content, hoping to ease the communication of an idea to the largest possible audience or to a specific audience. We consciously manipulate most of what is available to us, but not all. There is something else we can use. I often think that brilliant work demands a keen understanding of the magical visual language that informs design.
Embracing an established visual language
This is a language whose alphabet is shapes, structures, colours, lines and rhythms. When effective, it is somewhat invisible, subliminally enforcing messages and evoking meaning, using methods solidly rooted in a grammar perceptible in virtually all extraordinary creative work. The syntax for art, architecture, film, and furniture, industrial and graphic design (think Bauhaus and the Swiss style perhaps), this language urges us to become fluent if we aim for a more powerful dialogue with our audience.
Figure 1: Structures (clockwise from top-left): Informal; Formal; Active; Visible.
The greatest creative minds our world has produced could understand some or all of this language. Line and point, form and shape. Abstract objects. Formal and informal structures. Visual distribution. Balance, composition and the multitudinous approaches to symmetry. Patterns and texture. Movement and paths. Repetition, rhythm and frequency. Colour theory. Whitespace and the pause. The list goes on.
The genius we perceive in our creative heroes is often a composite of experience, trial and error, conviction, intuition – even accident – but rarely does great work arise without an initial understanding of the nuts and bolts that help communicate an idea or emotion.
Our world of interactivity
As web designers, our connection with this language is most evident in graphic design. With more technological ease and power comes the responsibility to understand, wisely use, and be able to justify many of our decisions. We have moved beyond the scope of print into a world of interactivity, but we shouldn’t let go of any established principles without good reason.
Figure 2: Understanding movement of objects in any direction along a defined path.
For example, immersion in this visual language can improve our implementation of CSS3 and JavaScript behaviour. With CSS3, we’ve seen a resurgence in CSS experimentation, some of which has been wonderful, but much of it has appeared clumsy. In the race to make something spin, twist, flip or fly from one corner to another, the designer sometimes fails to think about the true movement they seek to emulate. What forces are supposedly affecting this movement? What is the expected path of this transition and is it being respected?
Stopping to think about what is really supposed to be happening on the page compels us to use complex animations, diagrams and rotations more carefully. It helps us to better understand paths and movement.
Figure 3: Repetition can occur through variations in colour, shape, direction, and so on.
It can only be of greater benefit to be mindful of symmetries, depth, affordance, juxtaposition, balance, economy and reduction. A deeper understanding of basic structures can help us to say more with sketches, wireframes, layouts and composition. We’ve all experimented with grids and rhythm but, to truly benefit from these long-established principles, we are duty-bound to understand their possibilities more than we will by simply leveraging a free framework or borrowing some CSS.
Design is not a science, but…
Threading through all of this is what we have learned from science, and what it teaches us of the human brain. This visual language matters because technology changes but, for the most part, people don’t. For centuries, we humans have received and interpreted information in much the same way. Understanding more of how we perceive meaning can help designers make smarter decisions, and call on visual language to underpin these decisions. It is our responsibility as designers to be aware of mental models, mapping, semiotics, sensory experience and human emotion.
Design itself is not a science, but the appropriate use of visual language and scientific understanding exposes the line between effective and awkward, between communicative and mute. By strengthening our mental and analytical approach to what is often done arbitrarily or “because it feels right”, we simply become better designers.
A visual language for the web
So, I’ve outlined numerous starting points and areas worthy of deeper investigation, and hopefully you’re eager to do some research. However, I’ve mostly discussed established ideas and principles that we as web designers can learn from. It’s my belief that our community has a shared responsibility to expand this visual language as it applies to the ebb and flow of the web. Indulge me as I conclude with a related tangent.
In defining a visual language specifically for the web, we must continue to mature. The old powerfully influences the new, but we must intelligently expand the visual language of masterful work and articulate what is uniquely ours.
For example, phrases like Ethan Marcotte’s Responsive Web Design aren’t merely elegant, they describe a new way of thinking and working, of communicating about designs and interaction patterns. These phrases broaden our vocabulary and are immediately adopted by designers worldwide, in both conversation and execution.
Our legacy
Our new definitions should flex and not be tied to specific devices or methods which fade away or morph with time. Our legacy is perhaps more about robust and flexible patterns and systems than it is about specific devices or programming languages.
Figure 4: As web designers, we should think about systems, not pages.
The established principles we adopt and whatever new ways of thinking we define should slip neatly into a wider philosophy about our approach to web design. We’re called, as a community, to define what is distinctive about the visual language of the web, create this vocabulary, this dialect that resonates with us and moves us forward as we tackle each day’s work. Let’s give it some thought.
Further reading
This is my immediate “go-to” list of books that I bullishly believe all web designers should own, but there is so much more out there to read. Sadly, many great texts relating to this stuff are often out of print. Feel free to share your recommendations.
Don Norman, The Design of Everyday Things
Christian Leborg, Visual Grammar
Scott McCloud, Understanding Comics
David Crow, Visible Signs
William Lidwell and Katrina Holden, Universal Principles of Design",2010,Simon Collison,simoncollison,2010-12-16T00:00:00+00:00,https://24ways.org/2010/the-articulate-web-designer-of-tomorrow/,process
267,Taming Complexity,"I’m going to step into my UX trousers for this one. I wouldn’t usually wear them in public, but it’s Christmas, so there’s nothing wrong with looking silly.
Anyway, to business. Wherever I roam, I hear the familiar call for simplicity and the denouncement of complexity. I read often that the simpler something is, the more usable it will be. We understand that simple is hard to achieve, but we push for it nonetheless, convinced it will make what we build easier to use. Simple is better, right?
Well, I’ll try to explore that. Much of what follows will not be revelatory to some but, like all good lessons, I think this serves as a welcome reminder that as we live in a complex world it’s OK to sometimes reflect that complexity in the products we build.
Myths and legends
Less is more, we’ve been told, ever since master of poetic verse Robert Browning used the phrase in 1855. Well, I’ve conducted some research, and it appears he knew nothing of web design. Neither did modernist architect Ludwig Mies van der Rohe, a later pedlar of this worthy yet contradictory notion. Broad is narrow. Tall is short. Eggs are chips. See: anyone can come up with this stuff.
To paraphrase Einstein, simple doesn’t have to be simpler. In other words, simple doesn’t dictate that we remove the complexity. Complex doesn’t have to be confusing; it can be beautiful and elegant. On the web, complex can be necessary and powerful. A website that simplifies the lives of its users by offering them everything they need in one site or screen is powerful. For some, the greater the density of information, the more useful the site.
In our decision-making process, principles such as Occam’s razor’s_razor (in a nutshell: simple is better than complex) are useful, but simple is for the user to determine through their initial impression and subsequent engagement. What appears simple to me or you might appear very complex to someone else, based on their own mental model or needs. We can aim to deliver simple, but they’ll be the judge.
As a designer, developer, content alchemist, user experience discombobulator, or whatever you call yourself, you’re often wrestling with a wealth of material, a huge number of features, and numerous objectives. In many cases, much of that stuff is extraneous, and goes in the dustbin. However, it can be just as likely that there’s a truckload of suggested features and content because it all needs to be there. Don’t be afraid of that weight.
In the right hands, less can indeed mean more, but it’s just as likely that less can very often lead to, well… less.
Complexity is powerful
Simple is the ability to offer a powerful experience without overwhelming the audience or inducing information anxiety. Giving them everything they need, without having them ferret off all over a site to get things done, is important.
It’s useful to ask throughout a site’s lifespan, “does the user have everything they need?” It’s so easy to let our designer egos get in the way and chop stuff out, reduce down to only the things we want to see. That benefits us in the short term, but compromises the audience long-term.
The trick is not to be afraid of complexity in itself, but to avoid creating the perception of complexity. Give a user a flight simulator and they’ll crash the plane or jump out. Give them everything they need and more, but make it feel simple, and you’re building a relationship, empowering people.
This can be achieved carefully with what some call gradual engagement, and often the sensible thing might be to unleash complexity in carefully orchestrated phases, initially setting manageable levels of engagement and interaction, gradually increasing the inherent power of the product and fostering an empowered community.
The design aesthetic
Here’s a familiar scenario: the client or project lead gets overexcited and skips most of the important decision-making, instead barrelling straight into a bout of creative direction Tourette’s. Visually, the design needs to be minimal, white, crisp, full of white space, have big buttons, and quite likely be “clean”. Of course, we all like our websites to be clean as that’s more hygienic.
But what do these words even mean, really? Early in a project they’re abstract distractions, unnecessary constraints. This premature narrowing forces us to think much more about throwing stuff out rather than acknowledging that what we’re building is complex, and many of the components perhaps necessary.
Simple is not a formula. It cannot be achieved just by using a white background, by throwing things away, or by breathing a bellowsful of air in between every element and having it all float around in space. Simple is not a design treatment. Simple is hard. Simple requires deep investigation, a thorough understanding of every aspect of a project, in line with the needs and expectations of the audience.
Recognizing this helps us empathize a little more with those most vocal of UX practitioners. They usually appreciate that our successes depend on a thorough understanding of the user’s mental models and expected outcomes. I personally still consider UX people to be web designers like the rest of us (mainly to wind them up), but they’re web designers that design every decision, and by putting the user experience at the heart of their process, they have a greater chance of finding simplicity in complexity. The visual design aesthetic — the façade — is only a part of that.
Divide and conquer
I’m currently working on an app that’s complex in architecture, and complex in ambition. We’ll be releasing in carefully orchestrated private phases, gradually introducing more complexity in line with the unavoidably complex nature of the objective, but my job is to design the whole, the complete system as it will be when it’s out of beta and beyond.
I’ve noticed that I’m not throwing much out; most of it needs to be there. Therefore, my responsibility is to consider interesting and appropriate methods of navigation and bring everything together logically.
I’m using things like smart defaults, graphical timelines and colour keys to make sense of the complexity, techniques that are sympathetic to the content. They act as familiar points of navigation and reference, yet are malleable enough to change subtly to remain relevant to the information they connect. It’s really OK to have a lot of stuff, so long as we make each component work smartly.
It’s a divide and conquer approach. By finding simplicity and logic in each content bucket, I’ve made more sense of the whole, allowing me to create key layouts where most of the simplified buckets are collated and sometimes combined, providing everything the user needs and expects in the appropriate places.
I’m also making sure I don’t reduce the app’s power. I need to reflect the scale of opportunity, and provide access to or knowledge of the more advanced tools and features for everyone: a window into what they can do and how they can help. I know it’s the minority who will be actively building the content, but the power is in providing those opportunities for all.
Much of this will be familiar to the responsible practitioners who build websites for government, local authorities, utility companies, newspapers, magazines, banking, and we-sell-everything-ever-made online shops. Across the web, there are sites and tools that thrive on complexity.
Alas, the majority of such sites have done little to make navigation intuitive, or empower audiences. Where we can make a difference is by striving to make our UIs feel simple, look wonderful, not intimidating — even if they’re mind-meltingly complex behind that façade.
Embrace, empathize and tame
So, there are loads of ways to exploit complexity, and make it seem simple. I’ve hinted at some methods above, and we’ve already looked at gradual engagement as a way to make sense of complexity, so that’s a big thumbs-up for a release cycle that increases audience power.
Prior to each and every release, it’s also useful to rest on the finished thing for a while and use it yourself, even if you’re itching to release. ‘Ready’ often isn’t, and ‘finished’ never is, and the more time you spend browsing around the sites you build, the more you learn what to question, where to add, or subtract. It’s definitely worth building in some contingency time for sitting on your work, so to speak.
One thing I always do is squint at my layouts. By squinting, I get a sort of abstract idea of the overall composition, and general feel for the thing. It makes my face look stupid, but helps me see how various buckets fit together, and how simple or complex the site feels overall.
I mentioned the need to put our design egos to one side and not throw out anything useful, and I think that’s vital. I’m a big believer in economy, reduction, and removing the extraneous, but I’m usually referring to decoration, bells and whistles, and fluff. I wouldn’t ever advocate the complete removal of powerful content from a project roadmap.
Above all, don’t fear complexity. Embrace and tame it. Work hard to empathize with audience needs, and you can create elegant, playful, risky, surprising, emotive, delightful, and ultimately simple things.",2011,Simon Collison,simoncollison,2011-12-21T00:00:00+00:00,https://24ways.org/2011/taming-complexity/,ux
328,Swooshy Curly Quotes Without Images,"The problem
Take a quote and render it within blockquote tags, applying big, funky and stylish curly quotes both at the beginning and the end without using any images – at all.
The traditional way
Feint background images under the text, or an image in the markup housed in a little float. Often designers only use the opening curly quote as it’s just too difficult to float a closing one.
Why is the traditional way bad?
Well, for a start there are no actual curly quotes in the text (unless you’re doing some nifty image replacement). Thus with CSS disabled you’ll only have default blockquote styling to fall back on. Secondly, images don’t resize, so scaling text will have no affect on your graphic curlies.
The solution
Use really big text. Then it can be resized by the browser, resized using CSS, and even be restyled with a new font style if you fancy it. It’ll also make sense when CSS is unavailable.
The problem
Creating “Drop Caps” with CSS has been around for a while (Big Dan Cederholm discusses a neat solution in that first book of his), but drop caps are normal characters – the A to Z or 1 to 10 – and these can all be pulled into a set space and do not serve up a ton of whitespace, unlike punctuation characters.
Curly quotes aren’t like traditional characters. Like full stops, commas and hashes they float within the character space and leave lots of dead white space, making it bloody difficult to manipulate them with CSS. Styles generally fit around text, so cutting into that character is tricky indeed. Also, all that extra white space is going to push into the quote text and make it look pretty uneven. This grab highlights the actual character space:
See how this is emphasized when we add a normal alphabetical character within the span. This is what we’re dealing with here:
Then, there’s size. Call in a curly quote at less than 300% font-size and it ain’t gonna look very big. The white space it creates will be big enough, but the curlies will be way too small. We need more like 700% (as in this example) to make an impression, but that sure makes for a big character space.
Prepare the curlies
Firstly, remove the opening “ from the quote. Replace it with the opening curly quote character entity “. Then replace the closing “ with the entity reference for that, which is ”. Now at least the curlies will look nice and swooshy.
Add the hooks
Two reasons why we aren’t using :first-letter pseudo class to manipulate the curlies. Firstly, only CSS2-friendly browsers would get what we’re doing, and secondly we need to affect the last “letter” of our text also – the closing curly quote.
So, add a span around the opening curly, and a second span around the closing curly, giving complete control of the characters:
“ Speech marks. Curly quotes. That annoying thing cool people do with their fingers to emphasize a buzzword, shortly before you hit them.”
So far nothing will look any different, aside form the curlies looking a bit nicer. I know we’ve just added extra markup, but the benefits as far as accessibility are concerned are good enough for me, and of course there are no images to download.
The CSS
OK, easy stuff first. Our first rule .bqstart floats the span left, changes the color, and whacks the font-size up to an exuberant 700%. Our second rule .bqend does the same tricks aside from floating the curly to the right.
.bqstart {
float: left;
font-size: 700%;
color: #FF0000;
}
.bqend {
float: right;
font-size: 700%;
color: #FF0000;
}
That gives us this, which is rubbish. I’ve highlighted the actual span area with outlines:
Note that the curlies don’t even fit inside the span! At this stage on IE 6 PC you won’t even see the quotes, as it only places focus on what it thinks is in the div. Also, the quote text is getting all spangled.
Fiddle with margin and padding
Think of that span outline box as a window, and that you need to position the curlies within that window in order to see them. By adding some small adjustments to the margin and padding it’s possible to position the curlies exactly where you want them, and remove the excess white space by defining a height:
.bqstart {
float: left;
height: 45px;
margin-top: -20px;
padding-top: 45px;
margin-bottom: -50px;
font-size: 700%;
color: #FF0000;
}
.bqend {
float: right;
height: 25px;
margin-top: 0px;
padding-top: 45px;
font-size: 700%;
color: #FF0000;
}
I wanted the blocks of my curlies to align with the quote text, whereas you may want them to dig in or stick out more. Be aware however that my positioning works for IE PC and Mac, Firefox and Safari. Too much tweaking seems to break the magic in various browsers at various times. Now things are fitting beautifully:
I must admit that the heights, margins and spacing don’t make a lot of sense if you analyze them. This was a real trial and error job. Get it working on Safari, and IE would fail. Sort IE, and Firefox would go weird.
Finished
The final thing looks ace, can be resized, looks cool without styles, and can be edited with CSS at any time. Here’s a real example (note that I’m specifying Lucida Grande and then Verdana for my curlies):
“Speech marks. Curly quotes. That annoying thing cool people do with their fingers to emphasize a buzzword, shortly before you hit them.”
Browsers happy
As I said, too much tweaking of margins and padding can break the effect in some browsers. Even now, Firefox insists on dropping the closing curly by approximately 6 or 7 pixels, and if I adjust the padding for that, it’ll crush it into the text on Safari or IE. Weird. Still, as I close now it seems solid through resizing tests on Safari, Firefox, Camino, Opera and IE PC and Mac. Lovely.
It’s probably not perfect, but together we can beat the evil typographic limitations of the web and walk together towards a brighter, more aligned world. Merry Christmas.",2005,Simon Collison,simoncollison,2005-12-21T00:00:00+00:00,https://24ways.org/2005/swooshy-curly-quotes-without-images/,business
34,Collaborative Responsive Design Workflows,"Much has been written about workflow and designer-developer collaboration in web design, but many teams still struggle with this issue; either with how to adapt their internal workflow, or how to communicate the need for best practices like mobile first and progressive enhancement to their teams and clients. Christmas seems like a good time to have another look at what doesn’t work between us and how we can improve matters.
Why is it so difficult?
We’re still beginning to understand responsive design workflows, acknowledging the need to move away from static design tools and towards best practices in development. It’s not that we don’t want to change – so why is it so difficult?
Changing the way we do something that has become routine is always problematic, even with small things, and the changes today’s web environment requires from web design and development teams are anything but small.
Although developers also have a host of new skills to learn and things to consider, designers are probably the ones pushed furthest out of their comfort zones: as well as graphic design, a web designer today also needs an understanding of interaction design and ergonomics, because more and more websites are becoming tools rather than pages meant to be read like a book or magazine. In addition to that there are thousands of different devices and screen sizes on the market today that layout and interactions need to work on.
These aspects make it impossible to design in a static design tool, so beyond having to learn about new aspects of design, the designer has to either learn how to code or learn to work with a responsive design tool.
Why do it
That alone is enough to leave anyone overwhelmed, as learning a new skill takes time and slows you down in a project – and on most projects time is in short supply. Yet we have to make time or fall behind in the industry as others pitch better, interactive designs. For an efficient workflow, both designers and developers must familiarise themselves with new tools and techniques.
A designer has to be able to play with ideas, make small adjustments here and there, look at the result, go back to the settings and make further adjustments, and so on. You can only realistically do that if you are able to play with all the elements of a design, including interactivity, accessibility and responsiveness.
Figuring out the right breakpoints in a layout is one of the foremost reasons for designing in a responsive design tool. Even if you create layouts for three viewport sizes (i.e. smartphone, tablet and the most common desktop size), you’d only cover around 30% of visitors and you might miss problems like line breaks and padding at other viewport sizes.
Another advantage is consistency. In static design tools changes will not be applied across all your other layouts. A developer referring back to last week’s comps might work with outdated metrics. Furthermore, you cannot easily test what impact changes might have on previously designed areas. In a dynamic design tool such changes will be applied to the entire design and allow you to test things in site areas you had already finished.
No static design tool allows you to do this, and having somebody else produce a mockup from your static designs or wireframes will duplicate work and is inefficient.
How to do it
When working in a responsive design tool rather than in the browser, there is still the question of how and when to communicate with the developer. I have found that working with Sass in combination with a visual style guide is very efficient, but it does need careful planning: fundamental metrics for padding, margins and font sizes, but also design elements like sliders, forms, tabs, buttons and navigational elements, should be defined at the beginning of a project and used consistently across the site. Working with a grid can help you develop a consistent design language across your site.
Create a visual style guide that shows what the elements look like and how they behave across different screen sizes – and when interacted with. Put all metrics on paddings, margins, breakpoints, widths, colours and so on in a text document, ideally with names that your developer can use as Sass variables in the CSS. For example:
$padding-default-vertical: 1.5em;
Developers, too, need an efficient workflow to keep code maintainable and speed up the time needed for more complex interactions with an eye on accessibility and performance. CSS preprocessors like Sass allow you to work with variables and mixins for default rules, as well as style sheet partials for different site areas or design elements. Create your own boilerplate to use for your projects and then update your variables with the information from your designer for each individual project.
How to get buy-in
One obstacle when implementing responsive design, accessibility and content strategy is the logistics of learning new skills and iterating on your workflow. Another is how to sell it. You might expect everyone on a project (including the client) to want to design and develop the best website possible: ultimately, a great site will lead to more conversions. However, we often hear that people find it difficult to convince their teammates, bosses or clients to implement best practices.
Why is that? Well, I believe a lot of it is down to how we sell it. You will have experienced this yourself: some people you trust to know what they are talking about, and others you don’t. Think about why you trust that first person but don’t buy what the other one is telling you. It is likely because person A has a self-assured, calm and assertive demeanour, while person B seems insecure and apologetic. To sell our ideas, we need to become person A! For a timid designer or developer suffering from imposter syndrome (like many of us do in this industry) that is a difficult task. So how can we become more confident in selling our expertise?
Write
We need to become experts. And I mean not just in writing great code or coming up with beautiful designs but at explaining why we’re doing what we’re doing. Why do you code this way or that? Why is this the best layout? Why does a website have to be accessible and responsive? Write about it. Putting your thoughts down on paper or screen is a really efficient way of getting your head around a topic and learning to make a case for something. You may even find that you come up with new ideas as you are writing, so you’ll become a better designer or developer along the way.
Talk
Then, talk about it. Start out in front of your team, then do a lightning talk at a web event near you, then a longer talk or workshop. Having to talk about a topic is going to help you put into spoken words the argument that you’ve previously put together in writing. Writing comes more easily when you’re starting out but we use a different register when writing than talking and you need to learn how to speak your case. Do the talk a couple of times and after each talk make adjustments where you found it didn’t work well. By this time, you are more than ready to make your case to the client. In fact, you’ve been ready since that first talk in front of your colleagues ;)
Pitch
Pitches used to be based on a presentation of static layouts for for three to five typical pages and three different designs. But if we want to sell interactivity, structure, usability, accessibility and responsiveness, we need to demonstrate these things and I believe that it can only do us good. I have seen a few pitches sitting in the client’s chair and static layouts are always sort of dull. What makes a website a website is the fact that I can interact with it and smooth interactions or animations add that extra sparkle.
I can’t claim personal experience for this one but I’d be bold and go for only one design. One demo page matching the client’s corporate design but not any specific page for the final site. Include design elements like navigation, photography, typefaces, article layout (with real content), sliders, tabs, accordions, buttons, forms, tables (yes, tables) – everything you would include in a style tiles document, only interactive. Demonstrate how the elements behave when clicked, hovered and touched, and how they change across different screen sizes. You may even want to demonstrate accessibility features like tabbed navigation and screen reader use.
Obviously, there are many approaches that will work in different situations but don’t give up on finding a process that works for you and that ultimately allows you to build delightful, accessible, responsive user experiences for the web. Make time to try new tools and techniques and don’t just work on them on the side – start using them on an actual project. It is only when we use a tool or process in the real world that we become true experts. Remember your driving lessons: once the instructor had explained how to operate the car, you were sent to practise driving on the road in actual traffic!",2014,Sibylle Weber,sibylleweber,2014-12-07T00:00:00+00:00,https://24ways.org/2014/collaborative-responsive-design-workflows/,process
129,Knockout Type - Thin Is Always In,"OS X has gorgeous native anti-aliasing (although I will admit to missing 10px aliased Geneva — *sigh*). This is especially true for dark text on a light background. However, things can go awry when you start using light text on a dark background. Strokes thicken. Counters constrict. Letterforms fill out like seasonal snackers.
So how do we combat the fat? In Safari and other Webkit-based browsers we can use the CSS ‘text-shadow’ property. While trying to add a touch more contrast to the navigation on haveamint.com I noticed an interesting side-effect on the weight of the type.
The second line in the example image above has the following style applied to it:
This creates an invisible drop-shadow. (Why is it invisible? The shadow is positioned directly behind the type (the first two zeros) and has no spread (the third zero). So the color, black, is completely eclipsed by the type it is supposed to be shadowing.)
Why applying an invisible drop-shadow effectively lightens the weight of the type is unclear. What is clear is that our light-on-dark text is now of a comparable weight to its dark-on-light counterpart.
You can see this trick in effect all over ShaunInman.com and in the navigation on haveamint.com and Subtraction.com. The HTML and CSS source code used to create the example images used in this article can be found here.",2006,Shaun Inman,shauninman,2006-12-17T00:00:00+00:00,https://24ways.org/2006/knockout-type/,code
316,Have Your DOM and Script It Too,"When working with the XMLHttpRequest object it appears you can only go one of three ways:
You can stay true to the colorful moniker du jour and stick strictly to the responseXML property
You can play with proprietary – yet widely supported – fire and inject the value of responseText property into the innerHTML of an element of your choosing
Or you can be eval() and parse JSON or arbitrary JavaScript delivered via responseText
But did you know that there’s a fourth option giving you the best of the latter two worlds? Mint uses this unmentioned approach to grab fresh HTML and run arbitrary JavaScript simultaneously. Without relying on eval(). “But wait-”, you might say, “when would I need to do this?” Besides the example below this technique is handy for things like tab groups that need initialization onload but miss the main onload event handler by a mile thanks to asynchronous scripting.
Consider the problem
Originally Mint used option 2 to refresh or load new tabs into individual Pepper panes without requiring a full roundtrip to the server. This was all well and good until I introduced the new Client Mode which when enabled allows anyone to view a Mint installation without being logged in. If voyeurs are afoot as Client Mode is disabled, the next time they refresh a pane the entire login page is inserted into the current document. That’s not very helpful so I needed a way to redirect the current document to the login page.
Enter the solution
Wouldn’t it be cool if browsers interpreted the contents of script tags crammed into innerHTML? Sure, but unfortunately, that just wasn’t meant to be. However like the body element, image elements have an onload event handler. When the image has fully loaded the handler runs the code applied to it. See where I’m going with this?
By tacking a tiny image (think single pixel, transparent spacer gif – shudder) onto the end of the HTML returned by our Ajax call, we can smuggle our arbitrary JavaScript into the existing document. The image is added to the DOM, and our stowaway can go to town.
This is the results of our Ajax call.
Please be neat
So we’ve just jammed some meaningless cruft into our DOM. If our script does anything with images this addition could have some unexpected side effects. (Remember The Fly?) So in order to save that poor, unsuspecting element whose innerHTML we just swapped out from sharing Jeff Goldblum’s terrible fate we should tidy up after ourselves. And by using the removeChild method we do just that.
This is the results of our Ajax call.
",2005,Shaun Inman,shauninman,2005-12-24T00:00:00+00:00,https://24ways.org/2005/have-your-dom-and-script-it-too/,code
27,Putting Design on the Map,"The web can leave us feeling quite detached from the real world. Every site we make is really just a set of abstract concepts manifested as tools for communication and expression. At any minute, websites can disappear, overwritten by a newfangled version or simply gone. I think this is why so many of us have desires to create a product, write a book, or play with the internet of things. We need to keep in touch with the physical world and to prove (if only to ourselves) that we do make real things.
I could go on and on about preserving the web, the challenges of writing a book, or thoughts about how we can deal with the need to make real things. Instead, I’m going to explore something that gives us a direct relationship between a website and the physical world – maps.
A map does not just chart, it unlocks and formulates meaning; it forms bridges between here and there, between disparate ideas that we did not know were previously connected.
Reif Larsen, The Selected Works of T.S. Spivet
The simplest form of map on a website tends to be used for showing where a place is and often directions on how to get to it. That’s an incredibly powerful tool. So why is it, then, that so many sites just plonk in a default Google Map and leave it as that? You wouldn’t just use dark grey Helvetica on every site, would you? Where’s the personality? Where’s the tailored experience? Where is the design?
Jumping into design
Let’s keep this simple – we all want to be better web folk, not cartographers. We don’t need to go into the history, mathematics or technology of map making (although all of those areas are really interesting to research). For the sake of our sanity, I’m going to gloss over some of the technical areas and focus on the practical concepts.
Tiles
If you’ve ever noticed a map loading in sections, it’s because it uses tiles that are downloaded individually instead of requiring the user to download everything that they might need. These tiles come in many styles and can be used for anything that covers large areas, such as base maps and data. You’ve seen examples of alternative base maps when you use Google Maps as Google provides both satellite imagery and road maps, both of which are forms of base maps. They are used to provide context for the real world, or any other world for that matter. A marker on a blank page is useless.
The tiles are representations of the physical; they do not have to be photographic imagery to provide context. This means you can design the map itself. The easiest way to conceive this is by comparing Google’s road maps with Ordnance Survey road maps. Everything about the two maps is different: the colours, the label fonts and the symbols used. Yet they still provide the exact same context (other maps may provide different context such as terrain contours).
Comparison of Google Maps (top) and the Ordnance Survey (bottom).
Carefully designing the base map tiles is as important as any other part of the website. The most obvious, yet often overlooked, aspect are aesthetics and branding. Maps could fit in with the rest of the site; for example, by matching the colours and line weights, they can enhance the full design rather than inhibiting it. You’re also able to define the exact purpose of the map, so instead of showing everything you could specify which symbols or labels to show and hide.
I’ve not done any real research on the accessibility of base maps but, having looked at some of the available options, I think a focus on the typography of labels and the colour of the various elements is crucial. While you can choose to hide labels, quite often they provide the data required to make sense of the map. Therefore, make sure each zoom level is not too cluttered and shows enough to give context. Also be as careful when choosing the typeface as you are in any other design work. As for colour, you need to pay closer attention to issues like colour-blindness when using colour to convey information. Quite often a spectrum of colour will be used to show data, or to show the topography, so you need to be aware that some people struggle to see colour differences within a spectrum.
A nice example of a customised base map can be found on Michael K Owens’ check-in pages:
One of Michael K Owens’ check-in pages.
As I’ve already mentioned, tiles are not just for base maps: they are also for data. In the screenshot below you can see how Plymouth Marine Laboratory uses tiles to show data with a spectrum of colour.
A map from the Marine Operational Ecology data portal, showing data of adult cod in the North Sea.
Technical
You’re probably wondering how to design the base layers. I will briefly explain the concepts here and give you tools to use at the end of the article. If you’re worried about the time it takes to design the maps, don’t be – you can automate most of it. You don’t need to manually draw each tile for the entire world!
We’ve learned the importance of web standards the hard way, so you’ll be glad (and I won’t have to explain the advantages) of the standard for web mapping from the Open Geospatial Consortium (OGC) called the Web Map Service (WMS). You can use conventional file formats for the imagery but you need a way to query for the particular tiles to show for the area and zoom level, that is what WMS does.
Features
Tiles are great for covering large areas but sometimes you need specific smaller areas. We call these features and they usually consist of polygons, lines or points. Examples include postcode boundaries and routes between places, or even something more dynamic such as borders of nations changing over time.
Showing features on a map presents interesting design challenges. If the colour or shape conveys some kind of data beyond geographical boundaries then it needs to be made obvious. This is actually really hard, without building complicated user interfaces. For example, in the image below, is it obvious that there is a relationship between the colours? Does it need a way of showing what the colours represent?
Choropleth map showing ranked postcode areas, using ViziCities.
Features are represented by means of lines or colors; and the effective use of lines or colors requires more than knowledge of the subject – it requires artistic judgement.
Erwin Josephus Raisz, cartographer (1893–1968)
Where lots of boundaries are small and close together (such as a high street or shopping centre) will it be obvious where the boundaries are and what they represent? When designing maps, the hardest challenge is dealing with how the data is represented and how it is understood by the user.
Technical
As you probably gathered, we use WMS for tiles and another standard called the web feature service (WFS) for specific features. I need to stress that the difference between the two is that WMS is for tiling, whereas WFS is for specific features. Both can use similar file formats but should be used for their particular use cases. You may be wondering why you can’t just use a vector format such as KML, GeoJSON (or even SVG) – and you can – but the issue is the same as for WMS: you need a way to query the data to get the correct area and zoom level.
User interface
There is of course never a correct way to design an interface as there are so many different factors to take into consideration for each individual project. Maps can be used in a variety of ways, to provide simple information about directions or for complex visualisations to explain large amounts of data. I would like to just touch on matters that need to be taken into account when working with maps.
As I mentioned at the beginning, there are so many Google Maps on the web that people seem to think that its UI is the only way you can use a map. To some degree we don’t want to change that, as people know how to use them; but does every map require a zoom slider or base map toggle? In fact, does the user need to zoom at all? The answer to that one is generally yes, zooming does provide more context to where the map is zoomed in on.
In some cases you will need to let users choose what goes on the map (such as data layers or directions), so how do they show and hide the data? Does a simple drop-down box work, or do you need search? Google’s base map toggle is quite nice since it doesn’t offer many options yet provides very different contexts and styling.
It isn’t until we get to this point that we realise just plonking a quick Google map is really quite ridiculous, especially when compared to the amount of effort we make in other areas such as colour, typography or how the CSS is written. Each of these is important but we need to make sure the whole site is designed, and that includes the maps as much as any other content.
Putting it into practice
I could ramble on for ages about what we can do to customise maps to fit a site’s personality and correctly represent the data. I wanted to focus on concepts and standards because tools constantly change and it is never good to just rely on a tool to do the work. That said, there are a large variety of tools that will help you turn these concepts into reality. This is not a comparison; I just want to show you a few of the many options you have for maps on the web.
Google
OK, I’ve been quite critical so far about Google Maps but that is only because there is such a large amount of the default maps across the web. You can style them almost as much as anything else. They may not allow you to use custom WMS layers but Google Maps does have its own version, called styled maps. Using an array of map features (in the sense of roads and lakes and landmarks rather than the kind WFS is used for), you can style the base map with JavaScript. It even lets you toggle visibility, which helps to avoid the issue of too much clutter on the map. As well as lacking WMS, it doesn’t support WFS, but it does support GeoJSON and KML so you can still show the features on the map. You should also check out Google Maps Engine (the new version of My Maps), which provides an interface for creating more advanced maps with a selection of different base maps. A premium version is available, essentially for creating map-based visualisations, and it provides a step up from the main Google Maps offering. A useful feature in some cases is that it gives you access to many datasets.
Leaflet
You have probably seen Leaflet before. It isn’t quite as popular as Google Maps but it is definitely used often and for good reason. Leaflet is a lightweight open source JavaScript library. It is not a service so you don’t have to worry about API throttling and longevity. It gives you two options for tiling, the ability to use WMS, or to directly get the file using variables in the filename such as /{z}/{x}/{y}.png. I would recommend using WMS over dynamic file names because it is a standard, but the ability to use variables in a file name could be useful in some situations. Leaflet has a strong community and a well-documented API.
Mapbox
As a freemium service, Mapbox may not be perfect for every use case but it’s definitely worth looking into. The service offers incredible customisation tools as well as lots of data sources and hosting for the maps. It also provides plenty of libraries for the various platforms, so you don’t have to only use the maps on the web.
Mapbox is a service, though its map design tool is open source. Mapbox Studio is a vector-only version of their previous tool called Tilemill. Earlier I wrote about how typography and colour are as important to maps as they are to the rest of a website; if you thought, “Yes, but how on earth can I design those parts of a map?” then this is the tool for you. It is incredibly easy to use. Essentially each map has a stylesheet.
If you do not want to open a paid-for Mapbox account, then you can export the tiles (as PNG, SVG etc.) to use with other map tools.
OpenLayers
After a long wait, OpenLayers 3 has been released. It is similar to Leaflet in that it is a library not a service, but it has a much broader scope. During the last year I worked on the GIS portal at Plymouth Marine Laboratory (which I used to show the data tiles earlier), it essentially used OpenLayers 2 to create a web-based geographic information system, taking a large amount of data and permitting analysis (such as graphs) without downloading entire datasets and complicated software. OpenLayers 3 has improved greatly on the previous version in both performance and accessibility. It is the ideal tool for complex map-based web apps, though it can be used for the simple use cases too.
OpenStreetMap
I couldn’t write an article about maps on the web without at least mentioning OpenStreetMap. It is the place to go for crowd-sourced data about any location, with complete road maps and a strong API.
ViziCities
The newest project on this list is ViziCities by Robin Hawkes and Peter Smart. It is a open source 3-D visualisation tool, currently in the very early stages of development. The basic example shows 3-D buildings around the world using OpenStreetMap data. Robin has used it to create some incredible demos such as real-time London underground trains, and planes landing at an airport. Edward Greer and I are currently working on using ViziCities to show ideal housing areas based on particular personas. We chose it because the 3-D aspect gives us interesting possibilities for the data we are able to visualise (such as bar charts on the actual map instead of in the UI). Despite not being a completely stable, fully featured system, ViziCities is worth taking a look at for some use cases and is definitely going to go from strength to strength.
So there you have it – a whistle-stop tour of how maps can be customised. Now please stop plonking in maps without thinking about it and design them as you design the rest of your content.",2014,Shane Hudson,shanehudson,2014-12-11T00:00:00+00:00,https://24ways.org/2014/putting-design-on-the-map/,design
298,First Steps in VR,"The web is all around us. As web folk, it is our responsibility to consider the impact our work can have. Part of this includes thinking about the future; the web changes lives and if we are building the web then we are the ones making decisions that affect people in every corner of the world. I find myself often torn between wanting to make the right decisions, and just wanting to have fun. To fiddle and play. We all know how important it is to sometimes just try ideas, whether they will amount to much or not.
I think of these two mindsets as production and prototyping, though of course there are lots of overlap and phases in between. I mention this because virtual reality is currently seen as a toy for rich people, and in some ways at the moment it is. But with WebVR we are able to create interesting experiences with a relatively low entry point. I want us to have open minds, play around with things, and then see how we can use the tools we have at our disposal to make things that will help people.
Every year we see articles saying it will be the “year of virtual reality”, that was especially prevalent this year. 2016 has been a year of progress, VR isn’t quite mainstream but with efforts like Playstation VR and Google Cardboard, we are definitely seeing much more of it. This year also saw the consumer editions of the Oculus Rift and HTC Vive. So it does seem to be a good time for an overview of how to get involved with creating virtual reality on the web.
WebVR is an API for connecting to devices and retrieving continuous data such as the position and orientation. Unlike the Web Audio API and some other APIs, WebVR does not feel like a framework. You use it however you want, taking the data and using it as you wish. To make it easier, there are plenty of resources such as Three.js, A-Frame and ReactVR that help to make the heavy lifting a bit easier.
Getting Started with A-Frame
I like taking the opportunity to learn new things whenever I can. So while planning this article I thought that instead of trying to teach WebGL or even Three.js in a way that is approachable for all, I would create my first project using A-Frame and write about that. This is not a tutorial as such, I just want to show how to go about getting involved with VR. The beauty of A-Frame is that it is very similar to web components, you can just write HTML to build worlds that will automatically work on all the different types of devices. It uses WebGL and WebVR but in such a way that it quite drastically reduces the learning curve. That’s not to say you can’t build complex things, you have complete access to write JavaScript and shaders.
I’m lazy. Whenever I learn a new language or framework I have found that the best way, personally, for me to learn is to have a project and to copy the starting code from someone else. A project lets you have a good idea of what you want to produce and it means you can ignore a lot of the irrelevant documentation, focussing purely on what you need. That reduces the stress of figuring things out. Copying code also makes it easier, because you know your boilerplate code is working. There’s nothing worse than getting stuck before anything actually works the first time. So I tinker. I take code and I modify it, I play around. It’s fun.
For this project I wanted to keep things as simple as possible, so I can easily explain it without the classic “draw a circle then draw an owl”. I wrote a list of requirements, with some stretch goals that you can give a try yourself if you fancy:
Must work on Google Cardboard at a minimum, because of price
Therefore, it must not rely on having a controller
Auto-moving around a maze would be a good example
Move in direction you look
Stretch goal: Scoring, time until you hit a wall or get stuck in maze
Stretch goal: Levels, so the map doesn’t need to be random
Stretch goal: Snow!
I decided to base this project on an example, Platforms, by Don McCurdy who wrote the really useful aframe-extras. Platforms has random 3D blocks that you can jump onto, going up into the sky. So I took his code and reduced it so that the blocks are randomly spread on the ground.
24 ways
As you can see, this is very readable. Especially if you ignore the JavaScript that is used to create the maze. A-Frame (with A-Frame Extras) gives you a lot of power with relatively little to learn. We start with an which is the container for everything that is going to show up on the screen. There are a few which can be compared to as they are essentially non-semantic containers, able to be used for any purpose. The attributes are used to define functionality, for example the camera attribute sets the entity to function as a camera and kinematic-body makes it collide instead of go through objects. Attributes are also used to set position and sizes, often using JavaScript to dynamically define them.
Styling
Now we’ve got the HTML written, we need to style it. To do this we add A-Frame compatible attributes such as color and material. I recommend playing around, you can get some quite impressive effects fairly easily. Originally I wanted a light snowy maze but it ended up being dark and foggy, as I really liked the feeling it gave.
Note, you will probably need a server running for images to work. You can do this by running python -m ""SimpleHTTPServer"" in the folder where the code is, then go to localhost:8000 in browser.
Textures
Unless you are going for a cartoony style, you probably want to find some textures. I found some on textures.com, one image worked well for the walls and the other for the floor.
The
is used to define (as well as preload and cache) all assets, including images, audio and video. As you can see, images in the Asset Management System just use normal img tags. The ids are important here as we can use them later for using the textures.
To apply a texture to an object, you create a material. For a simple material where it just shows the image, you set the src to the id selector of the image.
Replace:
With:
This will automatically make the image repeat over the entire floor, in my case filling it with bricks. The walls are pretty much identical, with the slight exception that it is set in JavaScript as they are dynamically defined.
box.setAttribute('material', 'src: #texture-wall');
That’s it for the textures, for now at least. These will not look completely realistic, as the light will bump off the rectangular wall rather than texture itself. This can be improved by using maps, textures that are used to modify the shape and physical properties of the object.
Lighting
The next part of styling is lighting. By using fog and different types of lighting, we are able to add atmospheric details to the game to make it feel that bit more realistic and polished.
There are lots of types of light in A-Frame (most coming from Three.js). You can add a light either by using the entity or by attaching a light attribute to any other entity. If there are no lights defined then A-Frame adds some by default so that the scene is always lit.
To start with I wanted to light up the scene with a general light, type=""ambient"", so that the whole game felt slightly dark. I chose to set the light to a reddish colour #92455E. After playing around with intensity I chose 0.4, it added enough light to get the feeling I wanted without it being overly red. I also added a blue skybox (), as it looked a bit odd with a white sky.
I felt that the maze looked good with a red tinge but it was a bit flat, everything was the same colour and it was a bit dark. So I added a light within the #player entity, this could have been as an attribute but I set it as a child a-light instead. By using type=""point"" with a high intensity and low distance, it showed close walls as being lighter. It also added a sort-of object to the player, it isn’t a walking human or anything but by moving light where the player is it feels a bit more physical.
By this point it was starting to look decent, so I wanted to add the fog to really give some personality and depth to the maze. To do this I added the fog attribute to the with type=exponential so it looks thicker the further away it is and a mid intensity, so you feel a bit lost but can still see.
I was very happy with this result. It took a lot of playing around with colours and values, which is fun in itself. I highly recommend you take the code (or write your own) and play around with the numbers.
Movement
One of the reasons I decided to use aframe-extras is that it has a few different camera controls built in. As you saw earlier, I am using the universal-controls which gives WASD (keyboard) controls by default. I wanted to make it automatically move in the direction that you’re looking, but I wasn’t quite sure how without rewriting the controls. So I asked Don McCurdy for advice and he very nicely gave me a small snippet of code to get it working.
AFRAME.registerComponent('automove-controls', {
init: function () {
this.speed = 0.1;
this.isMoving = true;
this.velocityDelta = new THREE.Vector3();
},
isVelocityActive: function () {
return this.isMoving;
},
getVelocityDelta: function () {
this.velocityDelta.z = this.isMoving ? -speed : 0;
return this.velocityDelta.clone();
}
});
Replace:
universal-controls
With:
universal-controls=""movementControls: automove, gamepad, keyboard""
This works by creating a component automove-controls that adds auto-move to the player without overriding movement completely. It doesn’t even touch direction, it just checks if isMoving is true then moves the player by the set speed. Components can be creating for adding all kinds of functionality with relative ease. It makes it very powerful for people of all difficulty levels.
Building a map
Currently the maze is created randomly, which is great but means there will often be walls that overlap or the player gets trapped with nowhere to go. So to solve this, I decided to use a map editor (Tiled) so that we can create the mazes ourselves. This is a great start towards one of the stretch goals, levels.
I made the maze in Tiled by finding a random tileset online (we don’t need to actually show the images), I used one tile for the wall and another for the player. Then I exported as a JavaScript file and modified it in my text editor to get rid of everything I didn’t need. I made it so 0 is the path, 1 is the wall and 2 is the player. I then added the script to the HTML, as a separate file so it’s easy to update in the future.
var map =
{
""data"":[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
""height"":10,
""width"":10
}
As you can see, this gives a simple 10x10 maze with some dead ends. The player starts in the bottom right corner (my choice, could be anywhere). I rewrote the random platforms code (from Don’s example) to instead loop over the map data and place walls where it is 1 and position the player where data is 2. I set the position so that the origin of the map would be 0,1.5,0. The y axis is in this case the height (ground being 0), but if a wall is positioned at 0 by its centre then some of it is underground. So the y needed to be the height divided by 2.
document.querySelector('a-scene').addEventListener('render-target-loaded', function () {
var WALL_SIZE = 5,
WALL_HEIGHT = 3;
var el = document.querySelector('#walls');
var wall;
for (var x = 0; x < map.height; x++) {
for (var y = 0; y < map.width; y++) {
var i = y*map.width + x;
var position = (x-map.width/2)*WALL_SIZE + ' ' + 1.5 + ' ' + (y-map.height/2)*WALL_SIZE;
if (map.data[i] === 1) {
// Create wall
wall = document.createElement('a-box');
el.appendChild(wall);
wall.setAttribute('color', '#fff');
wall.setAttribute('material', 'src: #texture-wall;');
wall.setAttribute('width', WALL_SIZE);
wall.setAttribute('height', WALL_HEIGHT);
wall.setAttribute('depth', WALL_SIZE);
wall.setAttribute('position', position);
wall.setAttribute('static-body', ');
}
if (map.data[i] === 2) {
// Set player position
document.querySelector('#player').setAttribute('position', position);
}
}
}
console.info('Walls added.');
});
With this added, it makes it nice and easy to change around the map as well as to add new features. Perhaps you want monsters or objects. Just set the number in the map data and add an if statement to the loop. In the future you could add layers, so multiple things can be in the same position. Or perhaps even make the maze go up the y axis too, with ramps or staircases. There’s a lot you can do with relative ease. As you can see, A-Frame really does reduce the learning curve of 3D and VR on the web.
It’s Not All Fun And Games
A lot of examples of virtual reality are games, including this one. So it is understandable to think that VR is for gaming, but actually that’s just a tiny subset. There are all sorts of applications for VR, including story telling, data visualisation and even meditation.
There have been a number of cases where it has been shown virtual reality can help as a tool for therapies:
Oxford study finds virtual reality can help treat severe paranoia
Virtual Reality Therapy for Phobias at the Duke Faculty Practice
Bravemind: Virtual Reality Exposure Therapy at the University of Southern California
These are just a few examples of where virtual reality is being used around the world to help people feel better and get through some very tough times. There have also been examples of it being used for simulating war zones or medical situations, both as a teaching and journalism tool.
Wrapping Up
Ten years ago, on this very site, Cameron Moll wrote an article explaining the mobile web. He explained how mobile phones with data plans were becoming increasingly common, that WAP 2.0 included the XHTML Mobile Profile meaning it would be familiar with web folk. “The mobile web is rapidly becoming an XHTML environment, and thus you and I can apply our existing “desktop web” skills to understand how to develop content for it.”
We can look at that and laugh a little, we have come a very long way in the last decade. Even people in developing countries with very little money have mobile phones with access to a web that is far more capable than the “desktop web” Cameron was referring to.
So while I am not saying virtual reality is going to change the world or replace our phones, who knows! We can use our skills as web folk to dabble, we don’t need to learn any new languages. If on the 2026 edition of 24 ways, somebody references this article and looks at how far we have come… well, let’s hope we have used our skills well and made the world just that little bit better. And if VR is a fad? Well it’s fun… have a go anyway.",2016,Shane Hudson,shanehudson,2016-12-11T00:00:00+00:00,https://24ways.org/2016/first-steps-in-vr/,code
211,Automating Your Accessibility Tests,"Accessibility is one of those things we all wish we were better at. It can lead to a bunch of questions like: how do we make our site better? How do we test what we have done? Should we spend time each day going through our site to check everything by hand? Or just hope that everyone on our team has remembered to check their changes are accessible?
This is where automated accessibility tests can come in. We can set up automated tests and have them run whenever someone makes a pull request, and even alongside end-to-end tests, too.
Automated tests can’t cover everything however; only 20 to 50% of accessibility issues can be detected automatically. For example, we can’t yet automate the comparison of an alt attribute with an image’s content, and there are some screen reader tests that need to be carried out by hand too. To ensure our site is as accessible as possible, we will still need to carry out manual tests, and I will cover these later.
First, I’m going to explain how I implemented automated accessibility tests on Elsevier’s ecommerce pages, and share some of the lessons I learnt along the way.
Picking the right tool
One of the hardest, but most important parts of creating our automated accessibility tests was choosing the right tool.
We began by investigating aXe CLI, but soon realised it wouldn’t fit our requirements. It couldn’t check pages that required a visitor to log in, so while we could test our product pages, we couldn’t test any customer account pages. Instead we moved over to Pa11y. Its beforeScript step meant we could log into the site and test pages such as the order history.
The example below shows the how the beforeScript step completes a login form and then waits for the login to complete before testing the page:
beforeScript: function(page, options, next) {
// An example function that can be used to make sure changes have been confirmed before continuing to run Pa11y
function waitUntil(condition, retries, waitOver) {
page.evaluate(condition, function(err, result) {
if (result || retries < 1) {
// Once the changes have taken place continue with Pa11y testing
waitOver();
} else {
retries -= 1;
setTimeout(function() {
waitUntil(condition, retries, waitOver);
}, 200);
}
});
}
// The script to manipulate the page must be run with page.evaluate to be run within the context of the page
page.evaluate(function() {
const user = document.querySelector('#login-form input[name=""email""]');
const password = document.querySelector('#login-form input[name=""password""]');
const submit = document.querySelector('#login-form input[name=""submit""]');
user.value = 'user@example.com';
password.value = 'password';
submit.click();
}, function() {
// Use the waitUntil function to set the condition, number of retries and the callback
waitUntil(function() {
return window.location.href === 'https://example.com';
}, 20, next);
});
}
The waitUntil callback allows the test to be delayed until our test user is successfully logged in.
Another thing to consider when picking a tool is the type of error messages it produces. aXe groups all elements with the same error together, so the list of issues is a lot easier to read, and it’s easier to identify the most commons problems. For example, here are some elements that have insufficient colour contrast:
Violation of ""color-contrast"" with 8 occurrences!
Ensures the contrast between foreground and background colors meets
WCAG 2 AA contrast ratio thresholds. Correct invalid elements at:
- #maincontent > .make_your_mark > div:nth-child(2) > p > span > span
- #maincontent > .make_your_mark > div:nth-child(4) > p > span > span
- #maincontent > .inform_your_decisions > div:nth-child(2) > p > span > span
- #maincontent > .inform_your_decisions > div:nth-child(4) > p > span > span
- #maincontent > .inform_your_decisions > div:nth-child(6) > p > span > span
- #maincontent > .inform_your_decisions > div:nth-child(8) > p > span > span
- #maincontent > .inform_your_decisions > div:nth-child(10) > p > span > span
- #maincontent > .inform_your_decisions > div:nth-child(12) > p > span > span
For details, see: https://dequeuniversity.com/rules/axe/2.5/color-contrast
aXe also provides links to their site where they discuss the best way to fix the problem.
In comparison, Pa11y lists each individual error which can lead to a very verbose list. However, it does provide helpful suggestions of how to fix problems, such as suggesting an alternative shade of a colour to use:
• Error: This element has insufficient contrast at this conformance level.
Expected a contrast ratio of at least 4.5:1, but text in this element has a contrast ratio of 2.96:1.
Recommendation: change text colour to #767676.
⎣ WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail
⎣ #maincontent > div:nth-child(10) > div:nth-child(8) > p > span > span
⎣ Featured products:
Integrating into our build pipeline
We decided the perfect time to run our accessibility tests would be alongside our end-to-end tests. We have a Jenkins job that detects changes to our staging site and then triggers the end-to-end tests, and in turn our accessibility tests. Our Jenkins job retrieves the contents of a GitHub repository containing our Pa11y script file and npm package manifest.
Once Jenkins has cloned the repository, it installs any dependencies and executes the tests via:
npm install && npm test
Bundling the URLs to be tested into our test script means we don’t have a command line style test where we list each URL we wish to test in the Jenkins CLI. It’s an effective method but can also be cluttered, and obscure which URLs are being tested.
In the middle of the office we have a monitor displaying a Jenkins dashboard and from this we can see if the accessibility tests are passing or failing. Everyone in the team has access to the Jenkins logs and when the build fails they can see why and fix the issue.
Fixing the issues
As mentioned earlier, Pa11y can generate a long list of areas for improvement which can be very verbose and quite overwhelming. I recommend going through the list to see which issues occur most frequently and fix those first. For example, we initially had a lot of errors around colour contrast, and one shade of grey in particular. By making this colour darker, the number of errors decreased, and we could focus on the remaining issues.
Another thing I like to do is to tackle the quick fixes, such as adding alt text to images. These are small things that allow us to make an impact instantly, giving us time to fix more detailed concerns such as addressing tabindex issues, or speaking to our designers about changing the contrast of elements on the site.
Manual testing
Adding automated tests to check our site for accessibility is great, but as I mentioned earlier, this can only cover 20-50% of potential issues. To improve on this, we need to test by hand too, either by ourselves or by asking others.
One way we can test our site is to throw our mouse or trackpad away and interact with the site using only a keyboard. This allows us to check items such as tab order, and ensure menu items, buttons etc. can be used without a mouse. The commands may be different on different operating systems, but there are some great guides online for learning more about these.
It’s tempting to add alt text and aria-labels to make errors go away, but if they don’t make any sense, what use are they really? Using a screenreader we can check that alt text accurately represents the image. This is also a great way to double check that our ARIA roles make sense, and that they correctly identify elements and how to interact with them. When testing our site with screen readers, it’s important to remember that not all screen readers are the same and some may interact with our site differently to others.
Consider asking a range of people with different needs and abilities to test your site, too. People experience the web in numerous ways, be they permanent, temporary or even situational. They may interact with your site in ways you hadn’t even thought about, so this is a good way to broaden your knowledge and awareness.
Tips and tricks
One of our main issues with Pa11y is that it may find issues we don’t have the power to solve. A perfect example of this is the one pixel image Facebook injects into our site. So, we wrote a small function to go though such errors and ignore the ones that we cannot fix.
const test = pa11y({
....
hideElements: '#ratings, #js-bigsearch',
...
});
const ignoreErrors: string[] = [
' ',
'',
' '
];
const filterResult = result => {
if (ignoreErrors.indexOf(result.context) > -1) {
return false;
}
return true;
};
Initially we wanted to focus on fixing the major problems, so we added a rule to ignore notices and warnings. This made the list or errors much smaller and allowed us focus on fixing major issues such as colour contrast and missing alt text. The ignored notices and warnings can be added in later after these larger issues have been resolved.
const test = pa11y({
ignore: [
'notice',
'warning'
],
...
});
Jenkins gotchas
While using Jenkins we encountered a few problems. Sometimes Jenkins would indicate a build had passed when in reality it had failed. This was because Pa11y had timed out due to PhantomJS throwing an error, or the test didn’t go past the first URL. Pa11y has recently released a new beta version that uses headless Chrome instead of PhantomJS, so hopefully these issues will less occur less often.
We tried a few approaches to solve these issues. First we added error handling, iterating over the array of test URLs so that if an unexpected error happened, we could catch it and exit the process with an error indicating that the job had failed (using process.exit(1)).
for (const url of urls) {
try {
console.log(url);
let urlResult = await run(url);
urlResult = urlResult.filter(filterResult);
urlResult.forEach(result => console.log(result));
}
catch (e) {
console.log('Error:', e);
process.exit(1);
}
}
We also had issues with unhandled rejections sometimes caused by a session disconnecting or similar errors. To avoid Jenkins indicating our site was passing with 100% accessibility, when in reality it had not executed any tests, we instructed Jenkins to fail the job when an unhandled rejection or uncaught exception occurred:
process.on('unhandledRejection', (reason, p) => {
console.log('Unhandled Rejection at:', p, 'reason:', reason);
process.exit(1);
});
process.on('uncaughtException', (err) => {
console.log('Caught exception: ${err}n');
process.exit(1);
});
Now it’s your turn
That’s it! That’s how we automated accessibility testing for Elsevier ecommerce pages, allowing us to improve our site and make it more accessible for everyone. I hope our experience can help you automate accessibility tests on your own site, and bring the web a step closer to being accessible to all.",2017,Seren Davies,serendavies,2017-12-07T00:00:00+00:00,https://24ways.org/2017/automating-your-accessibility-tests/,code
295,Internet of Stranger Things,"This year I’ve been running a workshop about using JavaScript and Node.js to work with all different kinds of electronics on the Raspberry Pi. So especially for 24 ways I’m going to show you how I made a very special Raspberry Pi based internet connected project! And nothing says Christmas quite like a set of fairy lights connected to another dimension1.
What you’ll see
You can rig up the fairy lights in your home, with the scrawly letters written under each one. The people from the other side (i.e. the internet) will be able to write messages to you from their browser in real time. In fact why not try it now; check this web page. When you click the lights in your browser, my lights (and yours) will turn on and off in real life! (There may be a queue if there are lots of people accessing it, hit the “Send a message” button and wait your turn.)
It’s all done with JavaScript, using Node.js running on both the Raspberry Pi and on the server. I’m using WebSockets to communicate in real time between the browser, server and Raspberry Pi.
What you’ll need
Raspberry Pi any of the following models: Zero (will need straight male header pins soldered2 and Micro USB OTG adaptor), A+, B+, 2, or 3
Micro SD card at least 4Gb Class 10 speed3
Micro USB power supply at least 2A
USB Wifi dongle (unless you have a Pi 3 - that has wifi built in).
Addressable fairy lights
Logic level shifter (with pins soldered unless you want to do it!)
Breadboard
Jumper wires (3x male to male and 4x female to male)
Optional but recommended
Base board to hold the Pi and Breadboard (often comes with a breadboard!)
Find links for where to buy all of these items that goes along with this tutorial. The total price should be around $1004.
Setting up the Raspberry Pi
You’ll need to install the SD card for the Raspberry Pi. You’ll find a link to download a disk image on the support document, ready-made with the Raspbian version of Linux, along with Node.js and all the files you need. Download it and write it to the SD card using the fantastic free software Etcher5.
Next up you have to configure the wifi details on the SD card. If you plug the card into your computer you should see a drive called BOOT. There’s a text file on there called wpa_supplicant.conf. Open it up in your favourite text editor and replace mywifi and mypassword with your wifi details6.
network={
ssid=""mywifi""
psk=""mypassword""
}
Save the file, eject the card from your computer and plug it into the Raspberry Pi.
If you have a base board or holder for the Raspberry Pi, attach it now. Then connect the wifi USB dongle7 and power supply, but don’t plug it in yet!
Wiring!
Time to wire everything up!
First of all, push the Logic Level Converter into the middle of the breadboard:
Logic Level Converter
The logic level converter may be labelled differently from the one in the diagram but the pins are usually exactly the same internally. I would just make sure the pins marked HV (High Voltage) are on the bottom and LV (Low Voltage) are on the top.
Raspberry Pi pins only output 3.3v but the lights need 5v. That’s why we need the logic level converter in there to boost up the signal.
Connect the first two wires between the Raspberry Pi pins and the breadboard:
Note that the pins on the Raspberry Pi are male, so you need a female to male jumper wire to connect between them and the breadboard. The colours don’t have to match but it’s easier to follow (and check) if you use the same ones as in the diagram.
Then the next two:
This is what you should have so far:
Lights
Now to connect the lights! My ones have a connector with three holes in it that I can push jumper wires into, and hopefully yours will too! So I used the male-to-male jumper wires to connect them to the breadboard.
Make sure that you connect the right end of the lights, mine has a male connector at the wrong end so it’s impossible to do this, but double check.
Also make sure that the holes in the light connector are the same as mine. To do this, follow the wires from the connector to the first light and look at the circuit board inside. You should just about be able to make out the connections labelled + (sometimes 5V, V+ or VCC), GND (or ‘-’ or G) and DI (sometimes DIN for data in).
You can just about make out the +, DI and GND on this picture. Note that on the other side of the board there is a DO for data out - that’s what takes the data along to the chip in the next light. Make sure that you’re plugging into the data-in and not the data-out!
That’s it! Everything’s plugged in and ready to go! But before you plug power into your Pi, double check all your wires and make sure they’re exactly right! You could damage your Raspberry Pi if it is not wired correctly. So triple check!
The Moment of Truth!
Plug in the Raspberry Pi and wait around a minute or two for it to boot up. If all is well, the lights should strobe rainbow colours for one second - that’s your confirmation that it’s connected to my WebSocket server and ready to receive messages from the upside-down!
However, if the first light in the string is pulsing red, it means that you’re not connected to the internet. So check the Troubleshooting section of the support document. If it’s pulsing green then you’re connected to the internet but can’t connect to my server. It must have gone down. Sorry! The code will keep trying so leave it running and maybe it’ll come back up.
Rig up the lights!
Fix the lights up on the wall however you want, pins, nails, tape. I’ve used cable clips. Just be careful! I’m using a 50 light string so I’ve programmed it to use the lights at the end for the letters. That way I have just under half the string to extend down to the floor where I can keep the Raspberry Pi.
Check the photo here to see how the lights line up, note that there are spare unused lights in-between each row:
Now visit lights.seb.ly and you’ll see this :
If you’re the only one online you’ll have direct connection to the lights and any letter you click on will light up both in the browser and in real life. If there are other people there, you’ll need to click the button to join the queue and wait your turn.
How it works - the geeky details!
Electronics:
The pins on the Raspberry Pi are known as GPIO pins, general-purpose input/output. You can connect a wide variety of electronic components to them, LED lights, buttons, switches, and sensors. You can turn the power to the pins on and off using Node.js (or Python, if you prefer).
Addressable LEDs or “Neopixels”
We’re only using one GPIO pin on the Raspberry Pi (the other connections are 5V, 3.3V and ground) and that single pin is controlling all of the lights in the string. The code turns the pin on and off really fast in strictly timed morse-code-like dots and dashes to transmit binary data. The chips attached to each LED decode the binary and adjust the output to the LED accordingly. That chip then sends the data on to the next light in the string.
The chips on each light are the WS2811, part of the WS281x family that come in a multitude of different form factors and are often packaged with tiny LEDs in a single component. They are commonly referred to as Neopixels8 and I used them on my Laser Light Synths project.
Neopixels with the chip and the LED all in one - it’s the white square shaped component and the darker square inside is the chip. These are only 5mm wide!
A Laser Light Synth! Covered with around 800 super bright neopixels!
Logic Level Converter
The logic level converter is a really cheap and easy way to change the level from 3.3v to 5v and back again. You must be careful that you do not connect 5v into a GPIO pin or you will most likely damage the Raspberry Pi processor chip.
Power
Neopixels can often draw a lot of current so you need to be careful how you power them. I’ve measured the current draw from the string to be less than 800mA so you should be fine wired directly to the 5V output. But if you use more lights or have them all on really bright at once, you’ll need to use a separate 5V power supply. If you want to learn more, check out Adafruit’s Neopixel Uberguide.
Node.js
There are two Node.js apps running here, one on the Raspberry Pi and one on my server. You can see the code on my GitHub at github.com/sebleedelisle/stranger-lights for the Raspberry Pi and github.com/sebleedelisle/stranger-lights-server for the server. And they’re hosted on npm as stranger-lights and stranger-lights-server.
The server side code sets up a standard web server to deliver the HTML for the web interface. It also sets up a WebSocket server that allows for real-time communication between the browser and the server. This server code also manages the queue and who is in control of the lights at any given time.
WebSockets
I’m using the excellent Socket.io library to manage the WebSocket connection. Both the browser and the Raspberry Pi Node.js app connects to my WebSocket server.
When you click on a letter in the browser, a message is sent to the server, which forwards it to the connected Raspberry Pi clients and also all the web browsers9.
The Raspberry Pi code
The Node.js app runs automatically on startup, and I made this happen by adding this to the /etc/rc.local file:
node /home/pi/strangerthings/client.js > /dev/null &
Anything in the rc.local file gets executed when the Pi boots up and this line of code runs the Node.js app and routes its output to nowhere (ie /dev/null). The & means that it runs it in the background and doesn’t hold up the boot process.
Working with the Raspberry Pi headless
You might know that when a computer has no screen or keyboard, you would refer to it as “running headless”. So just like most web servers, you need to configure it over the network with ssh10. If you’re on a mac you can find your Pi on the network through the name raspberrypi.local11, otherwise you’ll need to find its IP address. There’s more on the guide to Remote Access instructions on the Raspberry Pi website. And if you’re very new to the terminal, I highly recommend this great online Linux command line tutorial.
Improvements
This is quite an early experiment and I’m sure I’ll discover lots of optimisations over the next few weeks, especially if the server gets a proper hammering today! But there are a few things you can do. Obviously I’ve just rigged up my lights with Post-it notes. It’d be a lot nicer to get a paint brush and try to recreate the Winona-in-a-manic-state text style.
Where next?
Finding quality resources about Node.js for electronics on the Pi can be somewhat hit and miss, but this is getting better all the time. Alternatively I am thinking about running some online courses, please let me know if that’s something you’d be interested in, or sign up to my mailing list at st4i.com.
There are many many more resources for the Raspberry Pi with Python (gpiozero is a good place to start), so if that language works for you, you’ll be spoilt for choice!
Also take a look at Arduino - it’s an incredibly popular platform for electronics and the internet is literally bursting with resources.
I hope you enjoyed this little foray into the world of JavaScript electronics on the Raspberry Pi! If you get this working at home please let me know! Tweet me at @seb_ly.
Not a particularly original idea, but I don’t think I’ve seen anyone do it quite like this before, ie using WebSockets, and Node.js on a Raspberry Pi. Other examples: Internet of Stranger Things, Strangerlights.com, and loads of examples on Instructables ↩︎
Video guide to soldering pins on to a Pi Zero and further soldering advice from Adafruit ↩︎
Slower cards will work but performance may suffer ↩︎
Or £5,000 in UK money. Sorry, Brexit joke :) ↩︎
You will need a card reader on your computer - most micro SD cards come with an adaptor that fits standard SD slots. ↩︎
SSID and password should be all that you need but you can see all the config options on this wpa supplicant guide ↩︎
Raspberry Pi Zero will require the OTG to USB adaptor to attach the wifi dongle ↩︎
Thanks to Adafruit who invented the term neopixels so we don’t have to refer to them as WS281x any more! ↩︎
So you can see other people sending messages in the browser ↩︎
ssh is short for Secure Shell and is a way to connect to a remote computer and type in it just like you would in the terminal. ↩︎
You can change this default hostname using raspi-config ↩︎",2016,Seb Lee-Delisle,sebleedelisle,2016-12-01T00:00:00+00:00,https://24ways.org/2016/internet-of-stranger-things/,code
221,"“Probably, Maybe, No”: The State of HTML5 Audio","With the hype around HTML5 and CSS3 exceeding levels not seen since 2005’s Ajax era, it’s worth noting that the excitement comes with good reason: the two specifications render many years of feature hacks redundant by replacing them with native features. For fun, consider how many CSS2-based rounded corners hacks you’ve probably glossed over, looking for a magic solution. These days, with CSS3, the magic is border-radius (and perhaps some vendor prefixes) followed by a coffee break.
CSS3’s border-radius, box-shadow, text-shadow and gradients, and HTML5’s , and are some of the most anticipated features we’ll see put to creative (ab)use as adoption of the ‘new shiny’ grows. Developers jumping on the cutting edge are using subsets of these features to little detriment, in most cases. The more popular CSS features are design flourishes that can degrade nicely, but the current audio and video implementations in particular suffer from a number of annoyances.
The new shiny: how we got here
Sound involves one of the five senses, a key part of daily life for most – and yet it has been strangely absent from HTML and much of the web by default. From a simplistic perspective, it seems odd that HTML did not include support for the full multimedia experience earlier, despite the CD-ROM-based craze of the early 1990s. In truth, standards like HTML can take much longer to bake, but eventually deliver the promise of a lowered barrier to entry, consistent implementations and shiny new features now possible ‘for free’ just about everywhere.
was introduced early and naturally to HTML, despite having some opponents at the time. Perhaps and were avoided, given the added technical complexity of decoding various multi-frame formats, plus the hardware and bandwidth limitations of the era. Perhaps there were quarrels about choosing a standard format or – more simply – maybe these elements just weren’t considered to be applicable to the HTML-based web at the time. In any event, browser plugins from programs like RealPlayer and QuickTime eventually helped to fill the in-page audio/video gap, handling and markup which pointed to .wav, .avi, .rm or .mov files. Suffice it to say, the experience was inconsistent at best and, on the standards side of the fence right now, so is HTML5 in terms of audio and video.
: the theory
As far as HTML goes, the code for is simple and logical. Just as with , a src attribute specifies the file to load. Pretty straightforward – sounds easy, right?
Download mysong.ogg ;
Ah, if only it were that simple. The first problem is that the OGG audio format, while ‘free’, is not supported by some browsers. Conversely, nor is MP3, despite being a de facto standard used in all kinds of desktop software (and hardware). In fact, as of November 2010, no single audio format is commonly supported across all major HTML5-enabled browsers.
What you end up writing, then, is something like this:
Download mysong.ogg or mysong.mp3
Keep in mind, this is only a ‘first class’ experience for the HTML5 case; also, for non-supported browsers, you may want to look at another inline player (object/embed, or a JavaScript plus Flash API) to have inline audio. You can imagine the added code complexity in the case of supporting ‘first class’ experiences for older browsers, too.
: the caveats
With , you typically don’t have to worry about format support – it just works – and that’s part of what makes a standard wonderful. JPEG, PNG, BMP, GIF, even TIFF images all render just fine if for no better reason, perhaps, than being implemented during the ‘wild west’ days of the web. The situation with today reflects a very different – read: business-aware – environment in 2010. (Further subtext: There’s a lot of [potential] money involved.) Regrettably, this is a collision of free and commercial interests, where the casualty is ultimately the user. Second up in the casualty list is you, the developer, who has to write additional code around this fragmented support.
The HTML5 audio API as implemented in JavaScript has one of the most un-computer-like responses I’ve ever seen, and inspired the title of this post. Calling new Audio().canPlayType('audio/mp3'), which queries the system for format support according to a MIME type, is supposed to return one of “probably”, “maybe”, or “no”. Sometimes, you’ll just get a null or empty string, which is also fun. A “maybe” response does not guarantee that a format will be supported; sometimes audio/mp3 gives “maybe,” but then audio/mpeg; codecs=""mp3"" will give a more-solid “probably” response. This can vary by browser or platform, too, depending on native support – and finally, the user may also be able to install codecs, extending support to include other formats. (Are you excited yet?)
Damn you, warring formats!
New market and business opportunities go hand-in-hand with technology developments. What we have here is certainly not failure to communicate; rather, we have competing parties shouting loudly in public in attempts to influence mindshare towards a de facto standard for audio and video. Unfortunately, the current situation means that at least two formats are effectively required to serve the majority of users correctly.
As it currently stands, we have the free and open source software camp of OGG Vorbis/WebM and its proponents (notably, Mozilla, Google and Opera in terms of browser makers), up against the non-free, proprietary and ‘closed’ camp of MP3 and MPEG4/HE-AAC/H.264 – which is where you’ll find commitments from Apple and Microsoft, among others. Apple is likely in with H.264 for the long haul, given its use of the format for its iTunes music store and video offerings.
It is generally held that H.264 is a technically superior format in terms of file size versus quality, but it involves intellectual property and, in many use cases, requires licensing fees. To be fair, there is a business model with H.264 and much has been invested in its development, but this approach is not often the kind that wins over the web. On that front, OGG/WebM may eventually win for being a ‘free’ format that does not involve a licensing scheme.
Closed software and tools ideologically clash with the open nature of the web, which exists largely thanks to free and open technology. Because of philosophical and business reasons, support for audio and video is fragmented across browsers adopting HTML5 features. It does not help that a large amount of audio and video currently exists in non-free MP3 and MPEG-4 formats. Adoption of and may be slowed, since it is more complex than and may feel ‘broken’ to developers when edge cases are encountered. Furthermore, the HTML5 spec does not mandate a single required format. The end result is that, as a developer, you must currently provide at least both MP3 and OGG, for example, to serve most existing HTML5-based user agents.
Transitioning to
There will be some growing pains as developers start to pick up the new HTML5 shiny, while balancing the needs of current and older agents that don’t support either or the preferred format you may choose (for example, MP3). In either event, Flash or other plugins can be used as done traditionally within HTML4 documents to embed and play the relevant audio.
The SoundManager 2 page player demo in action.
Ideally, HTML5 audio should be used whenever possible with Flash as the backup option. A few JavaScript/Flash-based audio player projects exist which balance the two; in attempting to tackle this problem, I develop and maintain SoundManager 2, a JavaScript sound API which transparently uses HTML5 Audio() and, if needed, Flash for playing audio files. The internals can get somewhat ugly, but the transition between HTML4 and HTML5 is going to be just that – and even with HTML5, you will need some form of format fall-back in addition to graceful degradation.
It may be safest to fall back to MP3/MP4 formats for inline playback at this time, given wide support via Flash, some HTML5-based browsers and mobile devices. Considering the amount of MP3/MP4 media currently available, it is wiser to try these before falling through to a traditional file download process.
Early findings
Here is a brief list of behavioural notes, annoyances, bugs, quirks and general weirdness I have found while playing with HTML5-based audio at time of writing (November 2010):
Apple iPad/iPhone (iOS 4, iPad 3.2+)
Only one sound can be played at a time. If a second sound starts, the first is stopped.
No auto-play allowed. Sounds follow the pop-up window security model and can only be started from within a user event handler such as onclick/touch, and so on. Otherwise, playback attempts silently fail.
Once started, a sequence of sounds can be created or played via the ‘finish’ event of the previous sound (for example, advancing through a playlist without interaction after first track starts).
iPad, iOS 3.2: Occasional ‘infinite loop’ bug seen where audio does not complete and stop at a sound’s logical end – instead, it plays again from the beginning. Might be specific to example file format (HE-AAC) encoded from iTunes.
Apple Safari, OS X Snow Leopard 10.6.5
Critical bug: Safari 4 and 5 intermittently fail to load or play HTML5 audio on Snow Leopard due to bug(s) in QuickTime X and/or other underlying frameworks. Known Apple ‘radar’ bug: bugs.webkit.org #32159 (see also, test case.) Amusing side note: Safari on Windows is fine.
Apple Safari, Windows
Food for thought: if you download “Safari” alone on Windows, you will not get HTML5 audio/video support (tested in WinXP). You need to download “Safari + QuickTime” to get HTML5 audio/video support within Safari. (As far as I’m aware, Chrome, Firefox and Opera either include decoders or use system libraries accordingly. Presumably IE 9 will use OS-level APIs.)
General Quirks
Seeking and loading, ‘progress’ events, and calculating bytes loaded versus bytes total should not be expected to be linear, as users can arbitrarily seek within a sound. It appears that some support for HTTP ranges exists, which adds a bit of logic to UI code. Browsers seem to vary slightly in their current implementations of these features.
The onload event of a sound may be of little relevance, if non-linear loading is involved (see above note re: seeking).
Interestingly (perhaps I missed it), the current spec does not seem to specify a panning or left/right channel mix option.
The preload attribute values may vary slightly between browsers at this time.
Upcoming shiny: HTML5 Audio Data API
With access to audio data, you can incorporate waveform and spectrum elements that make your designs react to music.
The HTML5 audio spec does a good job covering the basics of playback, but did not initially get into manipulation or generation of audio on-the-fly, something Flash has had for a number of years now. What if JavaScript could create, monitor and change audio dynamically, like a sort of audio element? With that kind of capability, many dynamic audio processing features become feasible and, when combined with other media, can make for some impressive demos.
What started as a small idea among a small group of audio and programming enthusiasts grew to inspire a W3C audio incubator group, and continued to establish the Mozilla Audio Data API. Contributors wrote a patch for Firefox which was reviewed and revised, and is now slated to be in the public release of Firefox 4. Some background and demos are also detailed in an article from the BBC R&D blog.
There are plenty of live demos to see, which give an impression of the new creative ideas this API enables. Many concepts are not new in themselves, but it is exciting to see this sort of thing happening within the native browser context.
Mozilla is not alone in this effort; the WebKit folks are also working on a JavaScriptAudioNode interface, which implements similar audio buffering and sample elements.
The future?
It is my hope that we’ll see a common format emerge in terms of support across the major browsers for both audio and video; otherwise, support will continue to be fragmented and mildly frustrating to develop for, and that can impede growth of the feature. It’s a big call, but if had lacked a common format back in the wild west era, I doubt the web would have grown to where it is today.
Complaints and nitpicks aside, HTML5 brings excellent progress on the browser multimedia front, and the first signs of native support are a welcome improvement given all audio and video previously relied on plugins. There is good reason to be excited. While there is room for more, support could certainly be much worse – and as tends to happen with specifications, the implementations targeting them should improve over time.
Note: Thanks to Nate Koechley, who suggested the Audio().canPlayType() response be part of the article title.",2010,Scott Schiller,scottschiller,2010-12-08T00:00:00+00:00,https://24ways.org/2010/the-state-of-html5-audio/,code
255,Inclusive Considerations When Restyling Form Controls,"I would like to begin by saying 2018 was the year that we, as developers, visual designers, browser implementers, and inclusive design and experience specialists rallied together and achieved a long-sought goal: We now have the ability to fully style form controls, across all modern browsers, while retaining their ease of declaration, native functionality and accessibility.
I would like to begin by saying all these things. However, they’re not true. I think we spent the year debating about what file extension CSS should be written in, or something. Or was that last year? Maybe I’m thinking of next year.
Returning to reality, styling form controls is more tricky and time consuming these days rather than flat out “hard”. In fact, depending on the length of the styling-leash a particular browser provides, there are controls you can style quite a bit. As for browsers with shorter leashes, there are other options to force their controls closer to the visual design you’re tasked to match.
However, when striving for custom styled controls, one must be careful not to forget about the inherent functionality and accessibility that many provide. People expect and deserve the products and services they use and pay for to work for them. If these services are visually pleasing, but only function for those who fit the handful of personas they’ve been designed for, then we’ve potentially deprived many people the experiences they deserve.
Quick level setting
Getting down to brass tacks, when creating custom styled form controls that should retain their expected semantics and functionality, we have to consider the following:
Many form elements can be styled directly through standard and browser specific selectors, as well as through some clever styling of markup patterns. We should leverage these native options before reinventing any wheels.
It is important to preserve the underlying semantics of interactive controls. We must not unintentionally exclude people who use assistive technologies (ATs) that rely on these semantics.
Make sure you test what you create. There is a lot of underlying complexity to form controls which may not be immediately apparent if they’re judged solely by their visual presentation in a single browser, or with limited AT testing.
Visually resetting and restyling form controls
Over the course of 2018, I worked on a project where I tested and reported on the accessibility impact of styling various form controls. In conducting my research, I reviewed many of the form controls available in HTML, testing to see how malleable they were to direct styling from standardized CSS selectors.
As I expected, controls such as the various text fields could be restyled rather easily. However, other controls like radio buttons and checkboxes, or sub-elements of special text fields like date, search, and number spinners were resistant to standard-based styling. These particular controls and their sub-elements required specific pseudo-elements to reset and allow for restyling of some of their default presentation.
See the Pen form control styling comparisons by Scott (@scottohara) on CodePen.
https://codepen.io/scottohara/pen/gZOrZm/
Over the years, the ability to directly style form controls has been something many people have clamored for. However, one should realize the benefits of being able to restyle some of these controls may involve more effort than originally anticipated.
If you want to restyle a control from the ground up, then you must also recreate any :active, :focus, and :hover states for the control—all those things that were previously taken care of by browsers. Not only that, but anything you restyle should also work with Windows High Contrast mode, styling for dark mode, and other OS-level settings that browser respect without you even realizing.
You ever try playing with the accessibility settings of your display on macOS, or similar Windows setting?
It is also worth mentioning that any browser prefixed pseudo-elements are not standardized CSS selectors. As MDN mentions at the top of their pages documenting these pseudo-elements:
Non-standard
This feature is non-standard and is not on a standards track. Do not use it on production sites facing the Web: it will not work for every user. There may also be large incompatibilities between implementations and the behavior may change in the future.
While this may be a deterrent for some, it’s my opinion the risks are often only skin-deep. By which I mean if a non-standard selector does change, the control may look a bit quirky, but likely won’t cease to function. A bug report which requires a CSS selector change can be an easy JIRA ticket to close, after all.
Can’t make it? Fake it.
Internet Explorer 11 (IE11) is still neck-and-neck with other browsers in vying for the number 2 spot in desktop browser share. Due to IE not recognizing vendor-prefixed appearance properties, some essential controls like checkboxes won’t render as intended.
Additionally, some controls like select boxes, file uploads, and sub-elements of date fields (calendar popups) cannot be modified by just relying on styling their HTML selectors alone. This means that unless your company designs and develops with a progressive enhancement, or graceful degradation mindset, you’ll need to take a different approach in styling.
Getting clever with markup and CSS
The following CodePen demonstrates how we can create a custom checkbox markup pattern. By mindfully utilizing CSS sibling selectors and positioning of the native control, we can create custom visual styling while also retaining the functionality and accessibility expectations of a native checkbox.
See the Pen Accessible Styled Native Checkbox by Scott (@scottohara) on CodePen.
https://codepen.io/scottohara/pen/RqEayN/
Customizing checkboxes by visually hiding the input and styling well-placed markup with sibling selectors may seem old hat to some. However, many variations of these patterns do not take into account how their method of visually hiding the checkboxes can create discovery issues for certain screen reader navigation methods. For instance, if someone is using a mobile device and exploring by touch, how will they be able to drag their finger over an input that has been reduced to a single pixel, or positioned off screen?
As we move away from the simplicity of declaring a single HTML element and using clever CSS and markup patterns to create restyled form controls, we increase the need for additional testing to ensure no expected behaviors are lost. In other words, what should work in theory may not work in practice when you introduce the various different ways people may engage with a form control. It’s worth remembering: what might be typical interactions for ourselves may be problematic if not impossible for others.
Limitations to cleverness
Creative coding will allow us to apply more consistent custom styles to some of the more problematic form controls. There will be a varied amount of custom markup, CSS, and sometimes JavaScript that will be needed to preserve the control’s inherent usability and accessibility for each control we take this approach to.
However, this method of restyling still doesn’t solve for the lack of feature parity across different browsers. Nor is it a means to account for controls which don’t have a native HTML element equivalent, such as a switch or multi-thumb range slider? Maybe there’s a control that calls for a visual design or proposed user experience that would require too much fighting with a native control’s behavior to be worth the level of effort to implement. Here’s where we need to take another approach.
Using ARIA when appropriate
Sometimes we have no other option than to roll up our sleeves and start building custom form controls from scratch. Fair warning though: just because we’re not leveraging a native HTML control as our foundation, it doesn’t mean we have carte blanche to throw semantics out the window. Enter Accessible Rich Internet Applications (ARIA).
ARIA is a set of attributes that can modify existing elements, or extend HTML to include roles, properties and states that aren’t native to the language. While divs and spans have no meaningful semantic information for us to leverage, with help from the ARIA specification and ARIA Authoring Practices we can incorporate these elements to help create the UI that we need while still following the first rule of Using ARIA:
If you can use a native HTML element or attribute with the semantics and behavior you require already built in, instead of re-purposing an element and adding an ARIA role, state or property to make it accessible, then do so.
By using these documents as guidelines, and testing our custom controls with people of various abilities, we can do our best to make sure a custom control performs as expected for as many people as possible.
Exceptions to the rule
One example of a control that allows for an exception to the first rule of Using ARIA would be a switch control.
Switches and checkboxes are similar components, in that they have both on/checked and off/unchecked states. However, checkboxes are often expected within the context of forms, or used to filter search queries on e-commerce sites. Switches are typically used to instantly enable or deactivate a particular setting at a component or app-based level, as this is their behavior in the native mobile apps in which they were popularized.
While a switch control could be created by visually restyling a checkbox, this does not automatically mean that the underlying semantics and functionality will match the visual representation of the control. For example, the following CodePen restyles checkboxes to look like a switch control, but the semantics of the checkboxes remain which communicate a different way of interacting with the control than what you might expect from a native switch control.
See the Pen Switch Boxes - custom styled checkboxes posing as switches by Scott (@scottohara) on CodePen.
https://codepen.io/scottohara/pen/XyvoeE/
By adding a role=""switch"" to these checkboxes, we can repurpose the inherent checked/unchecked states of the native control, it’s inherent ability to be focused by Tab key, and Space key to toggle state.
But while this is a valid approach to take in building a switch, how does this actually match up to reality?
Does it pass the test(s)?
Whether deconstructing form controls to fully restyle them, or leveraging them and other HTML elements as a base to expand on, or create, a non-native form control, building it is just the start. We must test that what we’ve restyled or rebuilt works the way people expect it to, if not better.
What we must do here is run a gamut of comparative tests to document the functionality and usability of native form controls. For example:
Is the control implemented in all supported browsers?
If not: where are the gaps? Will it be necessary to implement a custom solution for the situations that degrade to a standard text field?
If so: is each browser’s implementation a good user experience? Is there room for improvement that can be tested against the native baseline?
Test with multiple input devices.
Where the control is implemented, what is the quality of the user experience when using different input devices, such as mouse, touchscreen, keyboard, speech recognition or switch device, to name a few.
You’ll find some HTML5 controls (like date pickers and number spinners) have additional UI elements that may not be announced to AT, or even allow keyboard accessibility. Often these controls can be adjusted by other means, such as text entry, or using arrow keys to increase or decrease values. If restyling or recreating a custom version of a control like these, it may make sense to maintain these native experiences as well.
How well does the control take to custom styles?
If a control can be styled enough to not need to be rebuilt from scratch, that’s great! But make sure that there are no adverse affects on the accessibility of it. For instance, range sliders can be restyled and maintain their functionality and accessibility. However, elements like progress bars can be negatively affected by direct styling.
Always test with different browser and AT pairings to ensure nothing is lost when controls are restyled.
Do specifications match reality?
If recreating controls to get around native limitations, such as the inability to style the options of a select element, or requiring a Switch control which is not native to HTML, do your solutions match user expectations?
For instance, selects have unique picker interfaces on touch devices. And switches have varied levels of support for different browser and screen reader pairings. Test with real people, and check your analytics. If these experiences don’t match people’s expectations, then maybe another solution is in order?
Wrapping up
While styling form controls is definitely easier than it’s ever been, that doesn’t mean that it’s at all simple, nor will it likely ever be. The level of difficulty you’re going to face is going to depend entirely on what it is you’re hoping to style, add-on to, or recreate. And even if you build your custom control exactly to specification, you’ll still be reliant on browsers and assistive technologies being able to fully understand the component they’ve been presented.
Forms and their controls are an incredibly important part of what we need the Internet for. Paying bills, scheduling appointments, ordering groceries, renewing your license or even ordering gifts for the holidays. These are all important tasks that people should be able to complete with as little effort as possible. Especially since for some, completing these tasks online might be their only option.
2018 didn’t end up being the year we got full customization of form controls sorted out. But that’s OK. If we can continue to mindfully work with what we have, and instead challenge ourselves to follow inclusive design principles, well thought out Form Design Patterns, and solve problems with an accessibility first approach, we may come to realize that we can get along just fine without fully branded drop downs.
And hey. There’s always next year, right?",2018,Scott O'Hara,scottohara,2018-12-13T00:00:00+00:00,https://24ways.org/2018/inclusive-considerations-when-restyling-form-controls/,code
276,Your jQuery: Now With 67% Less Suck,"Fun fact: more websites are now using jQuery than Flash.
jQuery is an amazing tool that’s made JavaScript accessible to developers and designers of all levels of experience. However, as Spiderman taught us, “with great power comes great responsibility.” The unfortunate downside to jQuery is that while it makes it easy to write JavaScript, it makes it easy to write really really f*ing bad JavaScript. Scripts that slow down page load, unresponsive user interfaces, and spaghetti code knotted so deep that it should come with a bottle of whiskey for the next sucker developer that has to work on it.
This becomes more important for those of us who have yet to move into the magical fairy wonderland where none of our clients or users view our pages in Internet Explorer. The IE JavaScript engine moves at the speed of an advancing glacier compared to more modern browsers, so optimizing our code for performance takes on an even higher level of urgency.
Thankfully, there are a few very simple things anyone can add into their jQuery workflow that can clear up a lot of basic problems. When undertaking code reviews, three of the areas where I consistently see the biggest problems are: inefficient selectors; poor event delegation; and clunky DOM manipulation. We’ll tackle all three of these and hopefully you’ll walk away with some new jQuery batarangs to toss around in your next project.
Selector optimization
Selector speed: fast or slow?
Saying that the power behind jQuery comes from its ability to select DOM elements and act on them is like saying that Photoshop is a really good tool for selecting pixels on screen and making them change color – it’s a bit of a gross oversimplification, but the fact remains that jQuery gives us a ton of ways to choose which element or elements in a page we want to work with. However, a surprising number of web developers are unaware that all selectors are not created equal; in fact, it’s incredible just how drastic the performance difference can be between two selectors that, at first glance, appear nearly identical. For instance, consider these two ways of selecting all paragraph tags inside a with an ID.
$(""#id p"");
$(""#id"").find(""p"");
Would it surprise you to learn that the second way can be more than twice as fast as the first? Knowing which selectors outperform others (and why) is a pretty key building block in making sure your code runs well and doesn’t frustrate your users waiting for things to happen.
There are many different ways to select elements using jQuery, but the most common ways can be basically broken down into five different methods. In order, roughly, from fastest to slowest, these are:
$(""#id"");
This is without a doubt the fastest selector jQuery provides because it maps directly to the native document.getElementbyId() JavaScript method. If possible, the selectors listed below should be prefaced with an ID selector in conjunction with jQuery’s .find() method to limit the scope of the page that has to be searched (as in the $(""#id"").find(""p"") example shown above).
$(""p"");, $(""input"");, $(""form""); and so on
Selecting elements by tag name is also fast, since it maps directly to the native document.getElementsByTagname() method.
$("".class"");
Selecting by class name is a little trickier. While still performing very well in modern browsers, it can cause some pretty significant slowdowns in IE8 and below. Why? IE9 was the first IE version to support the native document.getElementsByClassName() JavaScript method. Older browsers have to resort to using much slower DOM-scraping methods that can really impact performance.
$(""[attribute=value]"");
There is no native JavaScript method for this selector to use, so the only way that jQuery can perform the search is by crawling the entire DOM looking for matches. Modern browsers that support the querySelectorAll() method will perform better in certain cases (Opera, especially, runs these searches much faster than any other browser) but, generally speaking, this type of selector is Slowey McSlowersons.
$("":hidden"");
Like attribute selectors, there is no native JavaScript method for this one to use. Pseudo-selectors can be painfully slow since the selector has to be run against every element in your search space. Again, modern browsers with querySelectorAll() will perform slightly better here, but try to avoid these if at all possible. If you must use one, try to limit the search space to a specific portion of the page: $(""#list"").find("":hidden"");
But, hey, proof is in the performance testing, right? It just so happens that said proof is sitting right here. Be sure to notice the class selector numbers beside IE7 and 8 compared to other browsers and then wonder how the people on the IE team at Microsoft manage to sleep at night. Yikes.
Chaining
Almost all jQuery methods return a jQuery object. This means that when a method is run, its results are returned and you can continue executing more methods on them. Rather than writing out the same selector multiple times over, just making a selection once allows multiple actions to be run on it.
Without chaining
$(""#object"").addClass(""active"");
$(""#object"").css(""color"",""#f0f"");
$(""#object"").height(300);
With chaining
$(""#object"").addClass(""active"").css(""color"", ""#f0f"").height(300);
This has the dual effect of making your code shorter and faster. Chained methods will be slightly faster than multiple methods made on a cached selector, and both ways will be much faster than multiple methods made on non-cached selectors. Wait… “cached selector”? What is this new devilry?
Caching
Another easy way to speed up your code that seems to be a mystery to developers is the idea of caching your selectors. Think of how many times you end up writing the same selector over and over again in any project. Every $("".element"") selector has to search the entire DOM each time, regardless of whether or not that selector had been previously run. Running the selection once and then storing the results in a variable means that the DOM only has to be searched once. Once the results of a selector have been cached, you can do anything with them.
First, run your search (here we’re selecting all of the
elements inside ):
var blocks = $(""#blocks"").find(""li"");
Now, you can use the blocks variable wherever you want without having to search the DOM every time.
$(""#hideBlocks"").click(function() {
blocks.fadeOut();
});
$(""#showBlocks"").click(function() {
blocks.fadeIn();
});
My advice? Any selector that gets run more than once should be cached. This jsperf test shows just how much faster a cached selector runs compared to a non-cached one (and even throws some chaining love in to boot).
Event delegation
Event listeners cost memory. In complex websites and apps it’s not uncommon to have a lot of event listeners floating around, and thankfully jQuery provides some really easy methods for handling event listeners efficiently through delegation.
In a bit of an extreme example, imagine a situation where a 10×10 cell table needs to have an event listener on each cell; let’s say that clicking on a cell adds or removes a class that defines the cell’s background color. A typical way that this might be written (and something I’ve often seen during code reviews) is like so:
$('table').find('td').click(function() {
$(this).toggleClass('active');
});
jQuery 1.7 has provided us with a new event listener method, .on(). It acts as a utility that wraps all of jQuery’s previous event listeners into one convenient method, and the way you write it determines how it behaves. To rewrite the above .click() example using .on(), we’d simply do the following:
$('table').find('td').on('click',function() {
$(this).toggleClass('active');
});
Simple enough, right? Sure, but the problem here is that we’re still binding one hundred event listeners to our page, one to each individual table cell. A far better way to do things is to create one event listener on the table itself that listens for events inside it. Since the majority of events bubble up the DOM tree, we can bind a single event listener to one element (in this case, the ) and wait for events to bubble up from its children. The way to do this using the .on() method requires only one change from our code above:
$('table').on('click','td',function() {
$(this).toggleClass('active');
});
All we’ve done is moved the td selector to an argument inside the .on() method. Providing a selector to .on() switches it into delegation mode, and the event is only fired for descendants of the bound element (table) that match the selector (td). With that one simple change, we’ve gone from having to bind one hundred event listeners to just one. You might think that the browser having to do one hundred times less work would be a good thing and you’d be completely right. The difference between the two examples above is staggering.
(Note that if your site is using a version of jQuery earlier than 1.7, you can accomplish the very same thing using the .delegate() method. The syntax of how you write the function differs slightly; if you’ve never used it before, it’s worth checking the API docs for that page to see how it works.)
DOM manipulation
jQuery makes it very easy to manipulate the DOM. It’s trivial to create new nodes, insert them, remove other ones, move things around, and so on. While the code to do this is simple to write, every time the DOM is manipulated, the browser has to repaint and reflow content which can be extremely costly. This is no more evident than in a long loop, whether it be a standard for() loop, while() loop, or jQuery $.each() loop.
In this case, let’s say we’ve just received an array full of image URLs from a database or Ajax call or wherever, and we want to put all of those images in an unordered list. Commonly, you’ll see code like this to pull this off:
var arr = [reallyLongArrayOfImageURLs];
$.each(arr, function(count, item) {
var newImg = ' ';
$('#imgList').append(newImg);
});
There are a couple of problems with this. For one (which you should have already noticed if you’ve read the earlier part of this article), we’re making the $(""#imgList"") selection once for each iteration of our loop. The other problem here is that each time the loop iterates, it’s adding a new to the DOM. Each of those insertions is going to be costly, and if our array is quite large then this could lead to a massive slowdown or even the dreaded ‘A script is causing this page to run slowly’ warning.
var arr = [reallyLongArrayOfImageURLs],
tmp = '';
$.each(arr, function(count, item) {
tmp += ' ';
});
$('#imgList').append(tmp);
All we’ve done here is create a tmp variable that each is added to as it’s created. Once our loop has finished iterating, that tmp variable will contain all of our list items in memory, and can be appended to our all in one go. Browsers work much faster when working with objects in memory rather than on screen, so this is a much faster, more CPU-cycle-friendly method of building a list.
Wrapping up
These are far from being the only ways to make your jQuery code run better, but they are among the simplest ones to implement. Though each individual change may only make a few milliseconds of difference, it doesn’t take long for those milliseconds to add up. Studies have shown that the human eye can discern delays of as few as 100ms, so simply making a few changes sprinkled throughout your code can very easily have a noticeable effect on how well your website or app performs. Do you have other jQuery optimization tips to share? Leave them in the comments and help make us all better.
Now go forth and make awesome!",2011,Scott Kosman,scottkosman,2011-12-13T00:00:00+00:00,https://24ways.org/2011/your-jquery-now-with-less-suck/,code
277,Raising the Bar on Mobile,"One of the primary challenges of designing for mobile devices is that screen real estate is often in limited supply. Through the advocacy of Luke W and others, we’ve drawn comfort from the idea that this constraint ends up benefiting users and designers alike, from obvious advantages like portability and reach, to influencing our content strategy decisions through focus and restraint. But that doesn’t mean we shouldn’t take advantage of every last pixel of that screen we can snag!
As anyone who has designed a website for use on a smartphone can attest, there’s an awful lot of space on mobile screens dedicated to browser functions that would be better off toggled out of view. Unfortunately, the visibility of some of these elements is beyond our control, such as the buttons fixed to the bottom of the viewport in iOS’s Safari and the WebOS browser. However, in many devices, the address bar at the top can be manually hidden, and its absence frees up enough pixel room for a large, impactful heading, a critical piece of navigation, or even just a little more white space to air things out.
So, as my humble contribution to this most festive of web publications, today I’ll dig into the approach I used to hide the address bar in a browser-agnostic fashion for sites like BostonGlobe.com, and the jQuery Mobile framework.
Surveying the land
First, let’s assess the chromes of some popular, current mobile browsers. For example purposes, the following screen-captures feature the homepage of the Boston Globe site, without any address-bar-hiding logic in place.
Note: these captures are just mockups – actual experience on these platforms may vary.
On the left is iOS5’s Safari (running on iPhone), and on the right is Windows Phone 7 (pre-Mango).
BlackBerry 7 (left), and Android 2.3 (right).
WebOS (left), Opera Mini (middle), and Opera Mobile (right).
Some browsers, such the default browsers on WebOS and BlackBerry 5, hide the bar automatically without any developer intervention, but many of them don’t. Of these, we can only manually hide the address bar on iOS Safari and Android (according to Opera Web Opener, Mike Taylor, some discussion is underway for support in Opera Mini and Mobile as well, which would be great!). This is unfortunate, but iOS and Android are incredibly popular, so let’s direct our focus there.
Great API, or greatest API?
As it turns out, iOS and Android not only allow you to hide the address bar, they use the same JavaScript method to do so, too (this shouldn’t be surprising, given that they are both WebKit browsers, but nothing expected happens in mobile). However, the method they use is not exactly intuitive. You might set out looking for a JavaScript API dedicated to this purpose, like, say, window.toolbar.hide(), but alas, to hide the address bar you need to use the window.scrollTo method!
window.scrollTo(0, 0);
The scrollTo method is not new, it’s just this particular use of it that is. For the uninitiated, scrollTo is designed to scroll a document to a particular set of coordinates, assuming the document is large enough to scroll to that spot. The method accepts two arguments: a left coordinate; and a top coordinate. It’s both simple and supported well pretty much everywhere. In iOS and Android, these coordinates are calculated from the top of the browser’s viewport, just below the address bar (interestingly, it seems that some platforms like BlackBerry 6 treat the top of the browser chrome as 0 instead, meaning the page content is closer to 20px from the top).
Anyway, by passing the coordinates 0, 0 to the scrollTo method, the browser will jump to the top of the page and pull the address bar out of view! Of course, if a quick call to scrollTo was all we need to do to hide the address bar in iOS and Android, this article would be pretty short, and nothing new. Unfortunately, the first issue we need to deal with is that this method alone will not usually do the trick: it must be called after the page has finished loading.
The browser gives us a load event for just that purpose, so we’ll wrap our scrollTo method in it and continue on our merry way! We’ll use the standard, addEventListener method to bind the the load event, passing arguments for event name load, and a callback function to execute when the event is triggered.
window.addEventListener(""load"",function() {
window.scrollTo(0, 0);
});
For the sake of preventing errors in those using browsers that don’t support addEventListener, such as Internet Explorer 8 and under, let’s make sure that method exists before we use it:
if( window.addEventListener ){
window.addEventListener(""load"",function() {
window.scrollTo(0, 0);
});
}
Now we’re getting somewhere, but we must also call the method after the load event’s default behavior has been applied. For this, we can use the setTimeout method, delaying its execution to after the load event has run its course.
if( window.addEventListener ){
window.addEventListener(""load"",function() {
setTimeout(function(){
window.scrollTo(0, 0);
}, 0);
});
}
Sweet sugar of Christmas! Hit this demo in iOS and watch that address bar drift up and away!
Not so fast…
We’ve got a little problem: the approach above does work in iOS but, in some cases, it works a little too well. In the process of applying this behavior, we’ve broken one of the primary tenets of responsible web development: don’t break the browser’s default behaviour. This usability rule of thumb is often violated by developers with even the best of intentions, from breaking the browser’s back button through unrecorded Ajax page refreshes, to fancy momentum touch scrolling scripts that can wreak havoc in all but the most sophisticated of devices. In this case, we’ve prevented the browser’s native support of deep-linking to sections of a page (a hash identifier in the URL matching a page element’s id attribute, for example, http://example.com#contact) from working properly, because our script always scrolls to the top.
To avoid this collision, we’ll need to detect whether a deep link, or hash, is present in the URL before applying our logic. We can do this by ensuring that the location.hash property is falsey:
if( !window.location.hash && window.addEventListener ){
window.addEventListener( ""load"",function() {
setTimeout(function(){
window.scrollTo(0, 0);
}, 0);
});
}
Still works great! And a quick test using a hash-based URL confirms that our script will not execute when a deep anchor is in play. Now iOS is looking sharp, and we’ve added our feature defensively to avoid conflicts.
Now, on to Android…
Wait. You didn’t expect that we could write code for one browser and be finished, right? Of course you didn’t. I mentioned earlier that Android uses the same method for getting rid of the scrollbar, but I left out the fact that the arguments it prefers vary slightly, but significantly, from iOS. Bah!
Differering from the earlier logic from iOS, to remove the address bar on Android’s default browser, you need to pass a Y coordinate of 1 instead of 0. Aside from being just plain odd, this is particularly unfortunate because to any other browser on the planet, 1px is a very real, however small, distance from the top of the page!
window.scrollTo( 0, 1 );
Looks like we’re going to need a fork…
R UA Android?
At this point, some developers might decide to simply not support this feature in Android, and more determined devs might decide that a quick check of the User Agent string would be a reliable way to determine the browser and tweak the scroll value accordingly. Neither of those decisions would be tragic, but in the spirit of cross-browser and future-friendly development, I’ll propose an alternative.
By this point, it should be clear that neither of the implementations above offer a particularly intuitive way to hide an address bar. As such, one might be skeptical that these approaches will stick around very long in their present state in either browser. Perhaps at some point, Android will decide to use 0 like iOS, making our lives a little easier, or maybe some new browser will decide to model their address bar hiding method after one of these implementations. In any case, detecting the User Agent only allows us to apply logic based on the known present, and in the world of mobile, let’s face it, the present is already the past.
Writing a check
In this next step of today’s technique, we’ll apply some logic to quickly determine the behavior model of the browser we’re using, then capitalize on that model – without caring which browser it happens to come from – by applying the appropriate scroll distance.
To do this, we’ll rely on a fortunate side effect of Android’s implementation, which is when you programatically scroll the page to 1 using scrollTo, Android will report that it’s still at 0 because oddly enough, it is! Of course, any other browser in this situation will report a scroll distance of 1. Thus, by scrolling the page to 1, then asking the browser its scroll distance, we can use this artifact of their wacky implementation to our advantage and scroll to the location that makes sense for the browser in play.
Getting the scroll distance
To pull off our test, we’ll need to ask the browser for its current scroll distance. The methods for getting scroll distance are not entirely standardized across popular browsers, so we’ll need to use some cross-browser logic. The following scroll distance function is similar to what you’d find in a library like jQuery. It checks the few common ways of getting scroll distance before eventually falling back to 0 for safety’s sake (that said, I’m unaware of any browsers that won’t return a numeric value from one of the first three properties).
// scrollTop getter
function getScrollTop(){
return scrollTop = window.pageYOffset ||
document.compatMode === ""CSS1Compat"" && document.documentElement.scrollTop ||
document.body.scrollTop || 0;
}
In order to execute that code above, the body object (referenced here as document.body) will need to be defined already, or we’ll risk an error. To determine that it’s defined, we can run a quick timer to execute code as soon as that object is defined and ready for use.
var bodycheck = setInterval(function(){
if( document.body ){
clearInterval( bodycheck );
//more logic can go here!!
}
}, 15 );
Above, we’ve defined a 15 millisecond interval called bodycheck that checks if document.body is defined and, if so, clears itself of running again. Within that if statement, we can extend our logic further to run other code, such as our check for the scroll distance, defined via the variable scrollTop below:
var scrollTop,
bodycheck = setInterval(function(){
if( document.body ){
clearInterval( bodycheck );
scrollTop = getScrollTop();
}
}, 15 );
With this working, we can immediately scroll to 1, then check the scroll distance when the body is defined. If the distance reports 1, we’re likely in a non-Android browser, so we’ll scroll back to 0 and clean up our mess.
window.scrollTo( 0, 1 );
var scrollTop,
bodycheck = setInterval(function(){
if( document.body ){
clearInterval( bodycheck );
scrollTop = getScrollTop();
window.scrollTo( 0, scrollTop === 1 ? 0 : 1 );
}
}, 15 );
Cashing in
All of the pieces are written now, so all we need to do is combine them with our previous logic for scrolling when the window is loaded, and we’ll have a cross-browser solution of which John Resig would be proud. Here’s our combined code snippet, with some formatting updates rolled in as well:
(function( win ){
var doc = win.document;
// If there’s a hash, or addEventListener is undefined, stop here
if( !location.hash && win.addEventListener ){
//scroll to 1
window.scrollTo( 0, 1 );
var scrollTop = 1,
getScrollTop = function(){
return win.pageYOffset || doc.compatMode = ""CSS1Compat"" && doc.documentElement.scrollTop || doc.body.scrollTop || 0;
},
//reset to 0 on bodyready, if needed
bodycheck = setInterval(function(){
if( doc.body ){
clearInterval( bodycheck );
scrollTop = getScrollTop();
win.scrollTo( 0, scrollTop = 1 ? 0 : 1 );
}
}, 15 );
win.addEventListener( “load”, function(){
setTimeout(function(){
//reset to hide addr bar at onload
win.scrollTo( 0, scrollTop === 1 ? 0 : 1 );
}, 0);
} );
}
})( this );
View code example
And with that, we’ve got a bunch more room to play with on both iOS and Android.
Break out the eggnog
…because we’re not done yet! In the spirit of making our script act more defensively, there’s still another use case to consider. It was essential that we used the window’s load event to trigger our scripting, but on pages with a lot of content, its use can come at a cost. Often, a user will begin interacting with a page, scrolling down as they read, before the load event has fired. In those situations, our script will jump the user back to the top of the page, resulting in a jarring experience.
To prevent this problem from occurring, we’ll need to ensure that the page has not been scrolled beyond a certain amount. We can add a simple check using our getScrollTop function again, this time ensuring that its value is not greater than 20 pixels or so, accounting for a small tolerance.
if( getScrollTop() < 20 ){
//reset to hide addr bar at onload
window.scrollTo( 0, scrollTop === 1 ? 0 : 1 );
}
And with that, we’re pretty well protected! Here’s a final demo.
The completed script can be found on Github (full source: https://gist.github.com/1183357 ). It’s MIT licensed. Feel free to use it anywhere or any way you’d like!
Your thoughts?
I hope this article provides you with a browser-agnostic approach to hiding the address bar that you can use in your own projects today. Perhaps alternatively, the complications involved in this approach convinced you that doing this well is more trouble than it’s worth and, depending on the use case, that could be a fair decision. But at the very least, I hope this demonstrates that there’s a lot of work involved in pulling off this small task in only two major platforms, and that there’s a real need for standardization in this area.
Feel free to leave a comment or criticism and I’ll do my best to answer in a timely fashion.
Thanks, everyone!
Some parting notes
I scream, you scream…
At the time of writing, I was not able to test this method on the latest Android 4.0 (Ice Cream Sandwich) build. According to Sencha Touch’s browser scorecard, the browser in 4.0 may have a different way of managing the address bar, so I’ll post in the comments once I get a chance to dig into it further.
Short pages get no love
Today’s technique only works when the page is as tall, or taller than, the device’s available screen height, so that the address bar may be scrolled out of view. On a short page, you might work around this issue by applying a minimum height to the body element ( body { min-height: 460px; } ), but given the variety of screen sizes out there, not to mention changes in orientation, it’s tough to find a value that makes much sense (unless you manipulate it with JavaScript).",2011,Scott Jehl,scottjehl,2011-12-20T00:00:00+00:00,https://24ways.org/2011/raising-the-bar-on-mobile/,design
231,Designing for iOS: Life Beyond Media Queries,"Although not a new phenomenon, media queries seem to be getting a lot attention online recently and for the right reasons too – it’s great to be able to adapt a design with just a few lines of CSS – but many people are relying only on them to create an iPhone-specific version of their website.
I was pleased to hear at FOWD NYC a few weeks ago that both myself and Aral Balkan share the same views on why media queries aren’t always going to be the best solution for mobile. Both of us specialise in iPhone design ourselves and we opt for a different approach to media queries. The trouble is, regardless of what you have carefully selected to be display:none; in your CSS, the iPhone still loads everything in the background; all that large imagery for your full scale website also takes up valuable mobile bandwidth and time.
You can greatly increase the speed of your website by creating a specific site tailored to mobile users with just a few handy pointers – media queries, in some instances, might be perfectly suitable but, in others, here’s what you can do.
Redirect your iPhone/iPod Touch users
To detect whether someone is viewing your site on an iPhone or iPod Touch, you can either use JavaScript or PHP.
The JavaScript
if((navigator.userAgent.match(/iPhone/i)) || (navigator.userAgent.match(/iPod/i))) {
if (document.cookie.indexOf(""iphone_redirect=false"") == -1) window.location = ""http://mobile.yoursitehere.com"";
}
The PHP
if(strstr($_SERVER['HTTP_USER_AGENT'],'iPhone') || strstr($_SERVER['HTTP_USER_AGENT'],'iPod'))
{
header('Location: http://mobile.yoursitehere.com');
exit();
}
Both of these methods redirect the user to a site that you have made specifically for the iPhone. At this point, be sure to provide a link to the full version of the website, in case the user wishes to view this and not be thrown into an experience they didn’t want, with no way back.
Tailoring your site
So, now you’ve got 320 × 480 pixels of screen to play with – and to create a style sheet for, just as you would for any other site you build. There are a few other bits and pieces that you can add to your code to create a site that feels more like a fully immersive iPhone app rather than a website.
Retina display
When building your website specifically tailored to the iPhone, you might like to go one step further and create a specific style sheet for iPhone 4’s Retina display. Because there are four times as many pixels on the iPhone 4 (640 × 960 pixels), you’ll find specifics such as text shadows and borders will have to be increased.
(Credit to Thomas Maier)
Prevent user scaling
This declaration, added into the , stops the user being able to pinch-zoom in and out of your design, which is perfect if you are designing to the exact pixel measurements of the iPhone screen.
Designing for orientation
As iPhones aren’t static devices, you’ll also need to provide a style sheet for horizontal orientation. We can do this by inserting some JavaScript into the as follows:
You can also specify orientation styles using media queries. This is absolutely fine, as by this point you’ll already be working with mobile-specific graphics and have little need to set a lot of things to display:none;
Remove the address and status bars, top and bottom
To give you more room on-screen and to make your site feel more like an immersive web app, you can place the following declaration into the of your document’s code to remove the address and status bars at the top and bottom of the screen.
Making the most of inbuilt functions
Similar to mailto: e-mail links, the iPhone also supports another two handy URI schemes which are great for enhancing contact details. When tapped, the following links will automatically bring up the appropriate call or text interface:
Call us
Text us
iPhone-specific Web Clip icon
Although I believe them to be fundamentally flawed, since they rely on the user bookmarking your site, iPhone Web Clip icons are still a nice touch. You need just two declarations, again in the of your document:
For iPhone 4 you’ll need to create a 114 × 114 pixels icon; for a non-Retina display, a 57 × 57 pixels icon will do the trick.
Precomposed
Apple adds its standard gloss ‘moon’ over the top of any icon. If you feel this might be too much for your particular icon and would prefer a matte finish, you can add precomposed to the end of the apple-touch-icon declaration to remove the standard gloss.
Wrapping up
Media queries definitely have their uses. They make it easy to build a custom experience for your visitor, regardless of their browser’s size. For more complex sites, however, or where you have lots of imagery and other content that isn’t necessary on the mobile version, you can now use these other methods to help you out. Remember, they are purely for presentation and not optimisation; for busy people on the go, optimisation and faster-running mobile experiences can only be a good thing.
Have a wonderful Christmas fellow Webbies!",2010,Sarah Parmenter,sarahparmenter,2010-12-17T00:00:00+00:00,https://24ways.org/2010/life-beyond-media-queries/,code
296,Animation in Design Systems,"Our modern front-end workflow has matured over time to include design systems and component libraries that help us stay organized, improve workflows, and simplify maintenance. These systems, when executed well, ensure proper documentation of the code available and enable our systems to scale with reduced communication conflicts.
But while most of these systems take a critical stance on fonts, colors, and general building blocks, their treatment of animation remains disorganized and ad-hoc. Let’s leverage existing structures and workflows to reduce friction when it comes to animation and create cohesive and performant user experiences.
Understand the importance of animation
Part of the reason we treat animation like a second-class citizen is that we don’t really consider its power. When users are scanning a website (or any environment or photo), they are attempting to build a spatial map of their surroundings. During this process, nothing quite commands attention like something in motion.
We are biologically trained to notice motion: evolutionarily speaking, our survival depends on it. For this reason, animation when done well can guide your users. It can aid and reinforce these maps, and give us a sense that we understand the UX more deeply. We retrieve information and put it back where it came from instead of something popping in and out of place.
“Where did that menu go? Oh it’s in there.”
For a deeper dive into how animation can connect disparate states, I wrote about the Importance of Context-Shifting in UX Patterns for CSS-Tricks.
An animation flow on mobile.
Animation also aids in perceived performance. Viget conducted a study where they measured user engagement with a standard loading GIF versus a custom animation. Customers were willing to wait almost twice as long for the custom loader, even though it wasn’t anything very fancy or crazy. Just by showing their users that they cared about them, they stuck around, and the bounce rates dropped.
14 second generic loading screen.22 second custom loading screen.
This also works for form submission. Giving your personal information over to an online process like a static form can be a bit harrowing. It becomes more harrowing without animation used as a signal that something is happening, and that some process is completing. That same animation can also entertain users and make them feel as though the wait isn’t as long.
Eli Fitch gave a talk at CSS Dev Conf called: “Perceived Performance: The Only Kind That Really Matters”, which is one of my favorite talk titles of all time. In it, he discussed how we tend to measure things like timelines and network requests because they are more quantifiable–and therefore easier to measure–but that measuring how a user feels when visiting the site is more important and worth the time and attention.
In his talk, he states “Humans over-estimate passive waits by 36%, per Richard Larson of MIT”. This means that if you’re not using animation to speed up how fast the wait time of a form submission loads, users are perceiving it to be much slower than the dev tools timeline is recording.
Reign it in
Unlike fonts, colors, and so on, we tend to add animation in as a last step, which leads to disorganized implementations that lack overall cohesion. If you asked a designer or developer if they would create a mockup or build a UI without knowing the fonts they were working with, they would dislike the idea. Not knowing the building blocks they’re working with means that the design can fall apart or the development can break with something so fundamental left out at the start. Good animation works the same way.
The first step in reigning in your use of animation is to perform an animation audit. Look at all the places you are using animation on your site, or the places you aren’t using animation but probably should. (Hint: perceived performance of a loader on a form submission can dramatically change your bounce rates.)
Not sure how to perform a good audit? Val Head has a great chapter on it in her book, Designing Interface Animations, which has of buckets of research and great ideas.
Even some beautiful component libraries that have animation in the docs make this mistake. You don’t need every kind of animation, just like you don’t need every kind of font. This bloats our code. Ask yourself questions like: do you really need a flip 180 degree animation? I can’t even conceive of a place on a typical UI where that would be useful, yet most component libraries that I’ve seen have a mixin that does just this.
Which leads to…
Have an opinion
Many people are confused about Material Design. They think that Material Design is Motion Design, mostly because they’ve never seen anyone take a stance on animation before and document these opinions well. But every time you use Material Design as your motion design language, people look at your site and think GOOGLE. Now that’s good branding.
By using Google’s motion design language and not your own, you’re losing out on a chance to be memorable on your own website.
What does having an opinion on motion look like in practice? It could mean you’ve decided that you never flip things. It could mean that your eases are always going to glide. In that instance, you would put your efforts towards finding an ease that looks “gliding” and pulling out any transform: scaleX(-1) animation you find on your site. Across teams, everyone knows not to spend time mocking up flipping animation (even if they’re working on an entirely different codebase), and to instead work on something that feels like it glides. You save time and don’t have to communicate again and again to make things feel cohesive.
Create good developer resources
Sometimes people don’t incorporate animation into a design system because they aren’t sure how, beyond the base hover states. All animation properties can be broken into interchangeable pieces. This allows developers and designers alike to mix and match and iterate quickly, while still staying in the correct language. Here are some recommendations (with code and a demo to follow):
Create timing units, similar to h1, h2, h3. In a system I worked on recently, I called these t1, t2, t3. T1 would be reserved for longer pieces, down to t5 which is a bit like h5 in that it’s the default (usually around .25 seconds or thereabouts).
Keep animation easings for entrance, exit, entrance emphasis and exit emphasis that people can commonly refer to. This, and the animation-fill-mode, are likely to be the only two properties that can be reused for the entrance and exit of the animation.
Use the animation-name property to define the keyframes for the animation itself. I would recommend starting with 5 or 6 before making a slew of them, and see if you need more. Writing 30 different animations might seem like a nice resource, but just like your color palette having too many can unnecessarily bulk up your codebase, and keep it from feeling cohesive. Think critically about what you need here.
See the Pen Modularized Animation for Component Libraries by Sarah Drasner (@sdras) on CodePen.
The example above is pared-down, but you can see how in a robust system, having pieces that are interchangeable cached across the whole system would save time for iterations and prototyping, not to mention make it easy to make adjustments for different feeling movement on the same animation easily.
One low hanging fruit might be a loader that leads to a success dialog. On a big site, you might have that pattern many times, so writing up a component that does only that helps you move faster while also allowing you to really zoom in and focus on that pattern. You avoid throwing something together at the last minute, or using a GIF, which are really heavy and mushy on retina. You can make singular pieces that look really refined and are reusable.
React and Vue Implementations are great for reusable components, as you can create a building block with a common animation pattern, and once created, it can be a resource for all. Remember to take advantage of things like props to allow for timing and easing adjustments like we have in the previous example!
Responsive
At the very least we should ensure that interaction also works well on mobile, but if we’d like to create interactions that take advantage of all of the gestures mobile has to offer, we can use libraries like zingtouch or hammer to work with swipe or multiple finger detection. With a bit of work, these can all be created through native detection as well.
Responsive web pages can specify initial-scale=1.0 in the meta tag so that the device is not waiting the required 300ms on the secondary tap before calling action. Interaction for touch events must either start from a larger touch-target (40px × 40px or greater) or use @media(pointer:coarse) as support allows.
Buy-in
Sometimes people don’t create animation resources simply because it gets deprioritized. But design systems were also something we once had to fight for, too. This year at CSS Dev Conf, Rachel Nabors demonstrated how to plot out animation wants vs. needs on a graph (reproduced with her permission) to help prioritize them:
This helps people you’re working with figure out the relative necessity and workload of the addition of these animations and think more critically about it. You’re also more likely to get something through if you’re proving that what you’re making is needed and can be reused.
Good compromises can be made this way: “we’re not going to go all out and create an animated ‘About Us’ page like you wanted, but I suppose we can let our users know their contact email went through with a small progress and success notification.”
Successfully pushing smaller projects through helps build trust with your team, and lets them see what this type of collaboration can look like. This builds up the type of relationship necessary to push through projects that are more involved. It can’t be overstressed that good communication is key.
Get started!
With these tools and good communication, we can make our codebases more efficient, performant, and feel better for our users. We can enhance the user experience on our sites, and create great resources for our teams to allow them to move more quickly while innovating beautifully.",2016,Sarah Drasner,sarahdrasner,2016-12-16T00:00:00+00:00,https://24ways.org/2016/animation-in-design-systems/,code
42,An Overview of SVG Sprite Creation Techniques,"SVG can be used as an icon system to replace icon fonts. The reasons why SVG makes for a superior icon system are numerous, but we won’t be going over them in this article. If you don’t use SVG icons and are interested in knowing why you may want to use them, I recommend you check out “Inline SVG vs Icon Fonts” by Chris Coyier – it covers the most important aspects of both systems and compares them with each other to help you make a better decision about which system to choose.
Once you’ve made the decision to use SVG instead of icon fonts, you’ll need to think of the best way to optimise the delivery of your icons, and ways to make the creation and use of icons faster.
Just like bitmaps, we can create image sprites with SVG – they don’t look or work exactly alike, but the basic concept is pretty much the same.
There are several ways to create SVG sprites, and this article will give you an overview of three of them. While we’re at it, we’re going to take a look at some of the available tools used to automate sprite creation and fallback for us.
Prerequisites
The content of this article assumes you are familiar with SVG. If you’ve never worked with SVG before, you may want to look at some of the introductory tutorials covering SVG syntax, structure and embedding techniques. I recommend the following:
SVG basics: Using SVG.
Structure: Structuring, Grouping, and Referencing in SVG — The , , and Elements. We’ll mention and quite a bit in this article.
Embedding techniques: Styling and Animating SVGs with CSS. The article covers several topics, but the section linked focuses on embedding techniques.
A compendium of SVG resources compiled by Chris Coyier — contains resources to almost every aspect of SVG you might be interested in.
And if you’re completely new to the concept of spriting, Chris Coyier’s CSS Sprites explains all about them.
Another important SVG feature is the viewBox attribute. For some of the techniques, knowing your way around this attribute is not required, but it’s definitely more useful if you understand – even if just vaguely – how it works. The last technique mentioned in the article requires that you do know the attribute’s syntax and how to use it. To learn all about viewBox, you can refer to my blog post about SVG coordinate systems.
With the prerequisites in place, let’s move on to spriting SVGs!
Before you sprite…
In order to create an SVG sprite with your icons, you’ll of course need to have these icons ready for use.
Some spriting tools require that you place your icons in a folder to which a certain spriting process is to be applied. As such, for all of the upcoming sections we’ll work on the assumption that our SVG icons are placed in a folder named SVG.
Each icon is an individual .svg file.
You’ll need to make sure each icon is well-prepared and optimised for use – make sure you’ve cleaned up the code by running it through one of the optimisation tools or processes available (or doing it manually if it’s not tedious).
After prepping the icon files and placing them in a folder, we’re ready to create our SVG sprite.
HTML inline SVG sprites
Since SVG is XML code, it can be embedded inline in an HTML document as a code island using the element. Chris Coyier wrote about this technique first on CSS-Tricks.
The embedded SVG will serve as a container for our icons and is going to be the actual sprite we’re going to use. So we’ll start by including the SVG in our document.
Next, we’re going to place the icons inside the . Each icon will be wrapped in a element we can then reference and use elsewhere in the page using the SVG element. The element has many benefits, and we’re using it because it allows us to define a symbol (which is a convenient markup for an icon) without rendering that symbol on the screen. The elements defined inside will only be rendered when they are referenced – or called – by the element.
Moreover, can have its own viewBox attribute, which makes it possible to control the positioning of its content inside its container at any time.
Before we move on, I’d like to shed some light on the style=""display:none;"" part of the snippet above. Without setting the display of the SVG to none, and even though its contents are not rendered on the page, the SVG will still take up space in the page, resulting in a big empty area. In order to avoid that, we’re hiding the SVG entirely with CSS.
Now, suppose we have a Twitter icon in the icons folder. twitter.svg might look something like this:
We don’t need the root svg element, so we’ll strip the code and only keep the parts that make up the Twitter icon’s shape, which in this example is just the element.Let’s drop that into the sprite container like so:
Repeat for the other icons.
The value of the element’s viewBox attribute depends on the size of the SVG. You don’t need to know how the viewBox works to use it in this case. Its value is made up of four parts: the first two will almost always be “0 0”; the second two will be equal to the size of the icon. For example, our Twitter icon is 32px by 32px (see twitter.svg above), so the viewBox value is “0 0 32 32”.
That said, it is certainly useful to understand how the viewBox works – it can help you troubleshoot SVG sometimes and gives you better control over it, allowing you to scale, position and even crop SVGs manually without having to resort to an editor. My blog post explains all about the viewBox attribute and its related attributes.
Once you have your SVG sprite ready, you can display the icons anywhere on the page by referencing them using the SVG element:
And that’s all there is to it!
HTML-inline SVG sprites are simple to create and use, but when you have a lot of icons (and the more icon sets you create) it can easily become daunting if you have to manually transfer the icons into the . Fortunately, you don’t have to do that. Fabrice Weinberg created a Grunt plugin called grunt-svgstore which takes the icons in your SVG folder and generates the SVG sprites for you; all you have to do is just drop the sprites into your page and use the icons like we did earlier.
This technique works in all browsers supporting SVG. There seems to be a bug in Safari on iOS which causes the icons not to show up when the SVG sprite is defined at the bottom of the document after the references to the icons, so it’s safest to include the sprite before you use the icons until this bug is fixed.
This technique has one disadvantage: the SVG sprite cannot be cached. We’re saving an extra HTTP request here but the browser cannot cache the image, so we aren’t speeding up any subsequent page loads by inlining the SVG. There must be a better way – and there is.
Styling the icons is possible, but getting deep into the styles becomes a bit harder owing to the nature of the contents of the element – these contents are cloned into a shadow DOM, and hence selecting elements in CSS the traditional way is not possible. However, some techniques to work around that do exist, and give us slightly more styling flexibility. Animations work as expected.
Referencing an external SVG sprite in HTML
Instead of including the SVG inline in the document, you can reference the sprite and the icons inside it externally, taking advantage of fragment identifiers to select individual icons in the sprite.
For example, the above reference to the Twitter icon would look something like this instead:
icons.svg is the name of the SVG file that contains all of our icons as symbols, and the fragment identifier #twitter-icon is the reference to the wrapping the Twitter icon’s contents. Very convenient, isn’t it? The browser will request the sprite and then cache it, speeding up subsequent page loads. Win!
This technique also works in all browsers supporting SVG except Internet Explorer – not even IE9+ with SVG support permits this technique. No version of IE supports referencing an external SVG in .
Fortunately (again), Jonathan Neil has created a plugin called svg4everybody which fills this gap in IE; you can reference an external sprite in and also provide fallback for browsers that do not support SVG. However, it requires you to have the fallback images (PNG or JPEG, for example) available to do so. For details, refer to the plugin’s Github repository’s readme file.
CSS inline SVG sprites
Another way to create an SVG sprite is by inlining the SVG icons in a style sheet using data URIs, and providing fallback for non-supporting browsers – also within the CSS.
Using this approach, we’re turning the style sheet into the sprite that includes our icons. The style sheet is normally cached by the browser, so we have that concern out of the way.
This technique is put into practice in Filament Group’s icon system approach, which uses their Grunticon plugin – or its sister Grumpicon web app – for generating the necessary CSS for the sprite. As such, we’re going to cover this technique by following a workflow that uses one of these tools.
Again, we start with our icon SVG files. To focus on the actual spriting method and not on the tooling, I’ll go over the process of sprite creation using the Grumpicon web app, instead of the Grunticon plugin. Both tools generate the same resources that we’re going to use for the icon system. Whether you choose the web app or the Grunt set-up, after processing your SVG folder you’re going to end up with the same set of resources that we’ll be using throughout this section.
The first step is to drop your icons into the Grumpicon web app.
Grumpicon homepage screenshot.
The application will then show you a preview of your icons, and a download button will allow you to download the generated files. These files will contain everything you need for your icon system – all that’s left is for you to drop the generated files and code into your project as recommended and you’ll have your sprite and icons ready to use anywhere you want in your page.
Grumpicon generates five files and one folder in the downloaded package: a png folder containing PNG versions of your icons; three style sheets (that we’ll go over briefly); a loader script file; and preview.html which is a live example showing you the other files in action.
The script in the loader goes into the of your page. This script handles browser and feature detection, and requests the necessary style sheet depending on browser support for SVG and base64 data URIs. If you view the source code of the preview page, you can see exactly how the script is added.
icons.data.svg.css is the style sheet that contains your icons – the sprite. The icons are embedded inline inside the style sheet using data URIs, and applied to elements of your choice as background images, using class names. For example:
.twitter-icon{
background-image: url('data:image/svg+xml;…'); /* the ellipsis is where the icon’s data would go */
background-repeat: no-repeat;
background-position: 50% 50%;
height: 2em;
width: 2em;
/* etc. */
}
Then, you only have to apply the twitter-icon class name to an element in your HTML to apply the icon as a background to it:
And that’s all you need to do to get an icon on the page.
icons.data.svg.css, along with the other two style sheets and the png folder should be added to your CSS folder.
icons.data.png.css is the style sheet the script will load in browsers that don’t support SVG, such as IE8. Fallback for the inline SVG is provided as a base64-encoded PNG. For instance, the fallback for the Twitter icon from our example would look like so:
.twitter-icon{
background-image: url('data:image/png;base64;…’);
/* etc. */
}
icons.fallback.css is the style sheet required for browsers that don’t support base64-encoded PNGs – the PNG images are loaded as usual using the image’s URL. The script will load this style sheet for IE6 and IE7, for example.
.twitter-icon{
background-image: url(png/twitter-icon.png);
/* etc. */
}
This technique is very different from the previous one. The sprite in this case is literally the style sheet, not an SVG container, and the icon usage is very similar to that of a CSS sprite – the icons are provided as background images.
This technique has advantages and disadvantages. For the sake of brevity, I won’t go into further details, but the main limitations worth mentioning are that SVGs embedded as background images cannot be styled with CSS; and animations are restricted to those defined inside the for each icon. CSS interactions (such as hover effects) don’t work either. Thus, to apply an effect for an icon that changes its color on hover, for example, you’ll need to export a set of SVGs for each colour in order for Grumpicon to create matching fallback PNG images that can then be used for the animation.
For more details about the Grumpicon workflow, I recommend you check out “A Designer’s Guide to Grumpicon” on Filament Group’s website.
Using SVG fragment identifiers and views
This spriting technique is, again, different from the previous ones, and it is my personal favourite.
SVG comes with a standard way of cropping to a specific area in a particular SVG image. If you’ve ever worked with CSS sprites before then this definitely sounds familiar: it’s almost exactly what we do with CSS sprites – the image containing all of the icons is cropped, so to speak, to show only the one icon that we want in the background positioning area of the element, using background size and positioning properties.
Instead of using background properties, we’ll be using SVG’s viewBox attribute to crop our SVG to the specific icon we want.
What I like about this technique is that it is more visual than the previous ones. Using this technique, the SVG sprite is treated like an actual image containing other images (the icons), instead of treating it as a piece of code containing other code.
Again, our SVG icons are placed inside a main SVG container that is going to be our SVG sprite. If you’re working in a graphics editor, position or arrange your icons inside the canvas any way you want them to be, and then export the graphic as is. Of course, the less empty space there is in your SVG, the better.
In our example, the sprite contains three icons as shown in the following image. The sprite is open in Sketch. Notice how the SVG is just big enough to fit the icons inside it. It doesn’t have to be like this, but it’s cleaner this way.
Screenshot showing the SVG sprite containing our icons.
Now, suppose you want to display only the Instagram icon. Using the SVG viewBox attribute, we can crop the SVG to the icon. The Instagram icon is positioned at 64px along the positive x-axis, and zero pixels along the y-axis. It is also 32px by 32px in size.
Screenshot showing the position (offset) of the Instagram icon inside the SVG sprite, and its size.
Using this information, we can specify the value of the viewBox as: 64 0 32 32. This area of the view box contains only the Instagram icon. 64 0 specifies the top-left corner of the view box area, and 32 32 specify its dimensions.
Now, if we were to change the viewBox value on the SVG sprite to this value, only the Instagram icon will be visible inside the SVG viewport. Great. But how do we use this information to display the icon in our page using our sprite?
SVG comes with a native way to link to portions or areas of an image using fragment identifiers. Fragment identifiers are used to link into a particular view area of an SVG document. Thus, using a fragment identifier and the boundaries of the area that we want (from the viewBox), we can link to that area and display it.
For example, if you want to display the icon from the sprite using an tag, you can reference the icon in the sprite like so:
The fragment identifier in the snippet above (#svgView(viewBox(64, 0, 32, 32))) is the important part. This will result in only the Instagram icon’s area of the sprite being displayed.
There is also another way to do this, using the SVG element. The element can be used to define a view area and then reference that area somewhere else. For example, to define the view box containing the Instagram icon, we can do the following:
Then, we can reference this view in our element like this:
The best part about this technique – besides the ability to reference an external SVG and hence make use of browser caching – is that it allows us to use practically any SVG embedding technique and does not restrict us to specific tags.
It goes without saying that this feature can be used for more than just icon systems, owing to viewBox’s power in controlling an SVG’s viewable area.
SVG fragment identifiers have decent browser support, but the technique is buggy in Safari: there is a bug that causes problems when loading a server SVG file and then using fragment identifiers with it. Bear Travis has documented the issue and a workaround.
Where to go from here
Pick the technique that works best for your project. Each technique has its own pros and cons, relating to convenience and maintainability, performance, and styling and scripting. Each technique also requires its own fallback mechanism.
The spriting techniques mentioned here are not the only techniques available. Other methods exist, such as SVG stacks, and others may surface in future, but these are the three main ones today.
The third technique using SVG’s built-in viewBox features is my favourite, and with better browser support and fewer (ideally, no) bugs, I believe it is more likely to become the standard way to create and use SVG sprites. Fallback techniques can be created, of course, in one of many possible ways.
Do you use SVG for your icon system? If so, which is your favourite technique? Do you know or have worked with other ways for creating SVG sprites?",2014,Sara Soueidan,sarasoueidan,2014-12-16T00:00:00+00:00,https://24ways.org/2014/an-overview-of-svg-sprite-creation-techniques/,code
30,"Making Sites More Responsive, Responsibly","With digital projects we’re used to shifting our thinking to align with our target audience. We may undertake research, create personas, identify key tasks, or observe usage patterns, with our findings helping to refine our ongoing creations. A product’s overall experience can make or break its success, and when it comes to defining these experiences our development choices play a huge role alongside more traditional user-focused activities.
The popularisation of responsive web design is a great example of how we are able to shape the web’s direction through using technology to provide better experiences. If we think back to the move from table-based layouts to CSS, initially our clients often didn’t know or care about the difference in these approaches, but we did. Responsive design was similar in this respect – momentum grew through the web industry choosing to use an approach that we felt would give a better experience, and which was more future-friendly.
We tend to think of responsive design as a means of displaying content appropriately across a range of devices, but the technology and our implementation of it can facilitate much more. A responsive layout not only helps your content work when the newest smartphone comes out, but it also ensures your layout suitably adapts if a visually impaired user drastically changes the size of the text.
The 24 ways site at 400% on a Retina MacBook Pro displays a layout more typically used for small screens.
When we think more broadly, we realise that our technical choices and approaches to implementation can have knock-on effects for the greater good, and beyond our initial target audiences. We can make our experiences more responsive to people’s needs, enhancing their usability and accessibility along the way.
Being responsibly responsive
Of course, when we think about being more responsive, there’s a fine line between creating useful functionality and becoming intrusive and overly complex. In the excellent Responsible Responsive Design, Scott Jehl states that:
A responsible responsive design equally considers the following throughout a project:
Usability: The way a website’s user interface is presented to the user, and how that UI responds to browsing conditions and user interactions.
Access: The ability for users of all devices, browsers, and assistive technologies to access and understand a site’s features and content.
Sustainability: The ability for the technology driving a site or application to work for devices that exist today and to continue to be usable and accessible to users, devices, and browsers in the future.
Performance: The speed at which a site’s features and content are perceived to be delivered to the user and the efficiency with which they operate within the user interface.
Scott’s book covers these ideas in a lot more detail than I’ll be able to here (put it on your Christmas list if it’s not there already), but for now let’s think a bit more about our roles as digital creators and the power this gives us.
Our choices around technology and the decisions we have to make can be extremely wide-ranging. Solutions will vary hugely depending on the needs of each project, though we can further explore the concept of making our creations more responsive through the use of humble web technologies.
The power of the web
We all know that under the HTML5 umbrella are some great new capabilities, including a number of JavaScript APIs such as geolocation, web audio, the file API and many more. We often use these to enhance the functionality of our sites and apps, to add in new features, or to facilitate device-specific interactions.
You’ll have seen articles with flashy titles such as “Top 5 JavaScript APIs You’ve Never Heard Of!”, which you’ll probably read, think “That’s quite cool”, yet never use in any real work.
There is great potential for technologies like these to be misused, but there are also great prospects for them to be used well to enhance experiences. Let’s have a look at a few examples you may not have considered.
Offline first
When we make websites, many of us follow a process which involves user stories – standardised snippets of context explaining who needs what, and why.
“As a student I want to pay online for my course so I don’t have to visit the college in person.”
“As a retailer I want to generate unique product codes so I can manage my stock.”
We very often focus heavily on what needs doing, but may not consider carefully how it will be done. As in Scott’s list, accessibility is extremely important, not only in terms of providing a great experience to users of assistive technologies, but also to make your creation more accessible in the general sense – including under different conditions.
Offline first is yet another ‘first’ methodology (my personal favourite being ‘tea first’), which encourages us to develop so that connectivity itself is an enhancement – letting users continue with tasks even when they’re offline. Despite the rapid growth in public Wi-Fi, if we consider data costs and connectivity in developing countries, our travel habits with planes, underground trains and roaming (or simply if you live in the UK’s signal-barren East Anglian wilderness as I do), then you’ll realise that connectivity isn’t as ubiquitous as our internet-addled brains would make us believe. Take a scenario that I’m sure we’re all familiar with – the digital conference. Your venue may be in a city served by high-speed networks, but after overloading capacity with a full house of hashtag-hungry attendees, each carrying several devices, then everyone’s likely to be offline after all. Wouldn’t it be better if we could do something like this instead?
Someone visits our conference website.
On this initial run, some assets may be cached for future use: the conference schedule, the site’s CSS, photos of the speakers.
When the attendee revisits the site on the day, the page shell loads up from the cache.
If we have cached content (our session timetable, speaker photos or anything else), we can load it directly from the cache. We might then try to update this, or get some new content from the internet, but the conference attendee already has a base experience to use.
If we don’t have something cached already, then we can try grabbing it online.
If for any reason our requests for new content fail (we’re offline), then we can display a pre-cached error message from the initial load, perhaps providing our users with alternative suggestions from what is cached.
There are a number of ways we can make something like this, including using the application cache (AppCache) if you’re that way inclined. However, you may want to look into service workers instead. There are also some great resources on Offline First! if you’d like to find out more about this.
Building in offline functionality isn’t necessarily about starting offline first, and it’s also perfectly possible to retrofit sites and apps to catch offline scenarios, but this kind of graceful degradation can end up being more complex than if we’d considered it from the start. By treating connectivity as an enhancement, we can improve the experience and provide better performance than we can when waiting to counter failures. Our websites can respond to connectivity and usage scenarios, on top of adapting how we present our content. Thinking in this way can enhance each point in Scott’s criteria.
As I mentioned, this isn’t necessarily the kind of development choice that our clients will ask us for, but it’s one we may decide is simply the right way to build based on our project, enhancing the experience we provide to people, and making it more responsive to their situation.
Even more accessible
We’ve looked at accessibility in terms of broadening when we can interact with a website, but what about how? Our user stories and personas are often of limited use. We refer in very general terms to students, retailers, and sometimes just users. What if we have a student whose needs are very different from another student? Can we make our sites even more usable and accessible through our development choices?
Again using JavaScript to illustrate this concept, we can do a lot more with the ways people interact with our websites, and with the feedback we provide, than simply accepting keyboard, mouse and touch inputs and displaying output on a screen.
Input
Ambient light detection is one of those features that looks great in simple demos, but which we struggle to put to practical use. It’s not new – many satnav systems automatically change the contrast for driving at night or in tunnels, and our laptops may alter the screen brightness or keyboard backlighting to better adapt to our surroundings. Using web technologies we can adapt our presentation to be better suited to ambient light levels.
If our device has an appropriate light sensor and runs a browser that supports the API, we can grab the ambient light in units using ambient light events, in JavaScript. We may then change our presentation based on different bandings, perhaps like this:
window.addEventListener('devicelight', function(e) {
var lux = e.value;
if (lux < 50) {
//Change things for dim light
}
if (lux >= 50 && lux <= 10000) {
//Change things for normal light
}
if (lux > 10000) {
//Change things for bright light
}
});
Live demo (requires light sensor and supported browser).
Soon we may also be able to do such detection through CSS, with light-level being cited in the Media Queries Level 4 specification. If that becomes the case, it’ll probably look something like this:
@media (light-level: dim) {
/*Change things for dim light*/
}
@media (light-level: normal) {
/*Change things for normal light*/
}
@media (light-level: washed) {
/*Change things for bright light*/
}
While we may be quick to dismiss this kind of detection as being a gimmick, it’s important to consider that apps such as Light Detector, listed on Apple’s accessibility page, provide important context around exactly this functionality.
“If you are blind, Light Detector helps you to be more independent in many daily activities. At home, point your iPhone towards the ceiling to understand where the light fixtures are and whether they are switched on. In a room, move the device along the wall to check if there is a window and where it is. You can find out whether the shades are drawn by moving the device up and down.”
everywaretechnologies.com/apps/lightdetector
Input can be about so much more than what we enter through keyboards. Both an ever increasing amount of available sensors and more APIs being supported by the major browsers will allow us to cater for more scenarios and respond to them accordingly. This can be as complex or simple as you need; for instance, while x-webkit-speech has been deprecated, the web speech API is available for a number of browsers, and research into sign language detection is also being performed by organisations such as Microsoft.
Output
Web technologies give us some great enhancements around input, allowing us to adapt our experiences accordingly. They also provide us with some nice ways to provide feedback to users.
When we play video games, many of our modern consoles come with the ability to have rumble effects on our controller pads. These are a great example of an enhancement, as they provide a level of feedback that is entirely optional, but which can give a great deal of extra information to the player in the right circumstances, and broaden the scope of our comprehension beyond what we’re seeing and hearing.
Haptic feedback is possible on the web as well. We could use this in any number of responsible applications, such as alerting a user to changes or using different patterns as a communication mechanism. If you find yourself in a pickle, here’s how to print out SOS in Morse code through the vibration API. The following code indicates the length of vibration in milliseconds, interspersed by pauses in milliseconds.
navigator.vibrate([100, 300, 100, 300, 100, 300, 600, 300, 600, 300, 600, 300, 100, 300, 100, 300, 100]);
Live demo (requires supported browser)
With great power…
What you’ve no doubt come to realise by now is that these are just more examples of progressive enhancement, whose inclusion will provide a better experience if the capabilities are available, but which we should not rely on. This idea isn’t new, but the most important thing to remember, and what I would like you to take away from this article, is that it is up to us to decide to include these kind of approaches within our projects – if we don’t root for them, they probably won’t happen. This is where our professional responsibility comes in.
We won’t necessarily be asked to implement solutions for the scenarios above, but they illustrate how we can help to push the boundaries of experiences. Maybe we’ll have to switch our thinking about how we build, but we can create more usable products for a diverse range of people and usage scenarios through the choices we make around technology. Let’s stop thinking simply in terms of features inside a narrow view of our target users, and work out how we can extend these to cater for a wider set of situations.
When you plan your next digital project, consider the power of the web and the enhancements we can use, and try to make your projects even more responsive and responsible.",2014,Sally Jenkinson,sallyjenkinson,2014-12-10T00:00:00+00:00,https://24ways.org/2014/making-sites-more-responsive-responsibly/,code
57,Cooking Up Effective Technical Writing,"Merry Christmas! May your preparations for this festive season of gluttony be shaping up beautifully. By the time you read this I hope you will have ordered your turkey, eaten twice your weight in Roses/Quality Street (let’s not get into that argument), and your Christmas cake has been baked and is now quietly absorbing regular doses of alcohol.
Some of you may be reading this and scoffing Of course! I’ve also made three batches of mince pies, a seasonal chutney and enough gingerbread men to feed the whole street! while others may be laughing Bake? Oh no, I can’t cook to save my life.
For beginners, recipes are the step-by-step instructions that hand-hold us through the cooking process, but even as a seasoned expert you’re likely to refer to a recipe at some point. Recipes tell us what we need, what to do with it, in what order, and what the outcome will be. It’s the documentation behind our ideas, and allows us to take the blueprint for a tasty morsel and to share it with others so they can recreate it. In fact, this is a little like the open source documentation and tutorials that we put out there, similarly aiming to guide other developers through our creations.
The ‘just’ification of documentation
Lately it feels like we’re starting to consider the importance of our words, and the impact they can have on others. Brad Frost warned us of the dangers of “Just” when it comes to offering up solutions to queries:
“Just use this software/platform/toolkit/methodology…”
“Just” makes me feel like an idiot. “Just” presumes I come from a specific background, studied certain courses in university, am fluent in certain technologies, and have read all the right books, articles, and resources. “Just” is a dangerous word.
“Just” by Brad Frost
I can really empathise with these sentiments. My relationship with code started out as many good web tales do, with good old HTML, CSS and JavaScript. University years involved some time with Perl, PHP, Java and C. In my first job I worked primarily with ColdFusion, a bit of ActionScript, some classic ASP and pinch of Java. I’d do a bit of PHP outside work every now and again. .NET came in, but we never really got on, and eventually I started learning some Ruby, Python and Node. It was a broad set of learnings, and I enjoyed the similarities and differences that came with new languages. I don’t develop day in, day out any more, and my interests and work have evolved over the years, away from full-time development and more into architecture and strategy. But I still make things, and I still enjoy learning.
I have often found myself bemoaning the lack of tutorials or courses that cater for the middle level – someone who may be learning a new language, but who has enough programming experience under their belt to not need to revise the concepts of how loops or objects work, and is perfectly adept at googling the syntax for getting a substring. I don’t want snippets out of context; I want an understanding of architectural principles, of the strengths and weaknesses, of the type of applications that work well with the language.
I’m caught in the place between snoozing off when ‘Using the Instagram API with Ruby’ hand-holds me through what REST is, and feeling like I’m stupid and need to go back to dev school when I can’t get my environment and dependencies set up, let alone work out how I’m meant to get any code to run.
It’s seems I’m not alone with this – Erin McKean seems to have been here too:
“Some tutorials (especially coding tutorials) like to begin things in media res. Great for a sense of dramatic action, bad for getting to “Step 1” without tears. It can be really discouraging to fire up a fresh terminal window only to be confronted by error message after error message because there were obligatory steps 0.1.0 through 0.9.9 that you didn’t even know about.”
“Tips for Learning What You Don’t Know You Don’t Know” by Erin McKean
I’m sure you’ve been here too. Many tutorials suffer badly from the fabled ‘how to draw an owl’-itis.
It’s the kind of feeling you can easily get when sifting through recipes as well as with code. Far from being the simple instructions that let us just follow along, they too can be a minefield. Fall in too low and you may be skipping over an explanation of what simmering is, or set your sights too high and you may get stuck at the point where you’re trying to sous vide a steak using your bathtub and a Ziploc bag.
Don’t be a turkey, use your loaf!
My mum is a great cook in my eyes (aren’t all mums?). I love her handcrafted collection of gathered recipes from over the years, including the one below, which is a great example of how something may make complete sense to the writer, but could be impermeable to a reader.
Depending on your level of baking knowledge, you may ask: What’s SR flour? What’s a tsp? Should I use salted or unsalted butter? Do I use sticks of cinnamon or ground? Why is chopped chocolate better? How do I cream things? How big should the balls be? How well is “well spaced”? How much leeway do I have for “(ish!!)”? Does the “20” on the other cookie note mean I’ll end up with twenty? At any point, making a wrong call could lead to rubbish cookies, and lead to someone heading down the path of an I can’t cook mentality.
You may be able to cook (or follow recipes), but you may not understand the local terms for ingredients, may not be able to acquire something and need to know what kind of substitutes you can use, or may need to actually do some prep before you jump into the main bit.
However, if we look at good examples of recipes, I think there’s a lot we can apply when it comes to technical writing on the web. I’ve written before about the benefit of breaking documentation into small, reusable parts, and this will help us, but we can also take it a bit further. Here are my five top tips for better technical writing.
1. Structure and standardise your information
Think of the structure of a recipe. We very often have some common elements and they usually follow roughly the same format. We have standards and conventions that allow us to understand very quickly what a recipe is and how it should be used.
Great recipes help their chefs know what they need to get ready in advance, both in terms of buying ingredients and putting together their kit. They then talk through the process, using appropriate language, and without making assumptions that the person can fill in any gaps for themselves; they explain why things are done the way they are. The best recipes may also suggest how you can take what you’ve done and put your own spin on it. For instance, a good recipe for the simple act of boiling an egg will explain cooking time in relation to your preference for yolk gooiness. There are also different flavour combinations to try, accompaniments, or presentation suggestions.
By breaking down your technical writing into similar sections, you can help your audience understand the elements they’ll be working with, what they need to do once they have these, and how they can move on from your self-contained illustration.
Title
Ensure your title is suitably descriptive and representative of the result. Getting Started with Python perhaps isn’t as helpful as Learn Python: General Syntax and Basics.
Result
Many recipes include a couple of lines as an overview of what you’ll end up with, and many include a photo of the finished dish. With our technical writing we can do the same:
In this tutorial we’re going to learn how to set up our development environment, and we’ll then undertake some exercises to explore the general syntax, finishing by building a mini calculator.
Ingredients
What are the components we’ll be working with, whether in terms of versions, environment, languages or the software packages and libraries you’ll need along the way? Listing these up front gives the reader a great summary of the things they’ll be using, and any gotchas.
Being able to provide a small amount of supporting information will also help less experienced users. Ideally, explain briefly what things are and why we’re using it.
Prep
As we heard from Erin above, not fully understanding the prep needed can be a huge source of frustration. Attempting to run a code snippet without context will often lead to failure when the prerequisites and process aren’t clear. Be sure to include information around any environment set-up, installation or config you’ll need to have done before you start.
Stu Robson’s Simple Sass documentation aims to do this before getting into specifics, although ideally this would also include setting up Sass itself.
Instructions
The body of the tutorial itself is the whole point of our writing. The next four tips will hopefully make your tutorial much more successful.
Variations
Like our ingredients section, as important as explaining why we’re using something in this context is, it’s also great to explain alternatives that could be used instead, and the impact of doing so.
Perhaps go a step further, explaining ways that people can change what you have done in your tutorial/readme for use in different situations, or to provide further reading around next steps. What happens if they want to change your static array of demo data to use JSON, for instance? By giving some thought to follow-up questions, you can better support your readers.
While not in a separate section, the source code for GreenSock’s GSAP JS basics explains:
We’ll use a window.onload for simplicity, but typically it is best to use either jQuery’s $(document).ready() or $(window).load() or cross-browser event listeners so that you’re not limited to one.
Keep in mind to both:
Explain what variations are possible.
Explain why certain options may be more desirable than others in different situations.
2. Small, reusable components
Reusable components are for life, not just for Christmas, and they’re certainly not just for development. If you start to apply the structure above to your writing, you’re probably going to keep coming across the same elements: Do I really have to explain how to install Sass and Node.js again, Sally? The danger with more clarity is that our writing becomes bloated and overly convoluted for advanced readers, those who don’t need to be told how to beat an egg for the hundredth time.
Instead, by making our writing reusable and modular, and by creating smaller, central resources, we can provide context and extra detail where needed without diluting our core message. These could be references we create, or those already created well by others.
This recipe for katsudon makes use of this concept. Rather than explaining how to make tonkatsu or dashi stock, these each have their own page. Once familiar, more advanced readers will likely skip over the instructions for the component parts.
3. Provide context to aid accessibility
Here I’m talking about accessibility in the broadest sense. Small, isolated snippets can be frustrating to those who don’t fully understand the wider context of how our examples work.
Showing an exciting standalone JavaScript function is great, but giving someone the full picture of how and when this is called, and how it should be included in relation to other HTML and CSS is even better. Giving your readers the ability to view a big picture version, and ideally the ability to download a full version of the source, will help to reduce some of the frustrations of trying to get your component to work in their set-up.
4. Be your own tech editor
A good editor can be invaluable to your work, and wherever possible I’d recommend that you try to get a neutral party to read over your writing. This may not always be possible, though, and you may need to rely on yourself to cast a critical eye over your work.
There are many tips out there around general editing, including printing out your work onto paper, or changing the font size: both will force your eyes to review it in a new light. Beyond this, I’d like to encourage you to think about the following:
Explain what things are. For example, instead of referencing Grunt, in the first instance perhaps reference “Grunt (a JavaScript task runner that minimises repetitive activities through automation).”
Explain how you get things, even if this is a link to official installers and documentation. Don’t leave your readers having to search.
Why are you using this approach/technology over other options?
What happens if I use something else? What depends on this?
Avoid exclusionary lingo or acronyms.
Airbnb’s JavaScript Style Guide includes useful pointers around their reasoning:
Use computed property names when creating objects with dynamic property names.
Why? They allow you to define all the properties of an object in one place.
The language we use often makes assumptions, as we saw with “just”. An article titled “ES6 for Beginners” is hugely ambiguous: is this truly for beginner coders, or actually for people who have a good pre-existing understanding of JavaScript but are new to these features? Review your writing with different types of readers in mind. How might you confuse or mislead them? How can you better answer their questions?
This doesn’t necessarily mean supporting everyone – your audience may need to have advanced skills – but even if you’re providing low-level, deep-dive, reference material, trying not to make assumptions or take shortcuts will hopefully lead to better, clearer writing.
5. A picture is worth a thousand words…
…or even better: use a thousand pictures, stitched together into a quick video or animated GIF. People learn in different ways. Just as recipes often provide visual references or a video to work along with, providing your technical information with alternative demonstrations can really help get your point across. Your audience will be able to see exactly what you’re doing, what they should expect as interaction responses, and what the process looks like at different points.
There are many, many options for recording your screen, including QuickTime Player on Mac OS X (File → New Screen Recording), GifGrabber, or Giffing Tool on Windows.
Paul Swain, a UX designer, uses GIFs to provide additional context within his documentation, improving communication:
“My colleagues (from across the organisation) love animated GIFs. Any time an interaction is referenced, it’s accompanied by a GIF and a shared understanding of what’s being designed. The humble GIF is worth so much more than a thousand words; and it’s great for cats.”
Paul Swain
Next time you’re cooking up some instructions for readers, think back to what we can learn from recipes to help make your writing as accessible as possible. Use structure, provide reusable bitesize morsels, give some context, edit wisely, and don’t scrimp on the GIFs. And above all, have a great Christmas!",2015,Sally Jenkinson,sallyjenkinson,2015-12-18T00:00:00+00:00,https://24ways.org/2015/cooking-up-effective-technical-writing/,content
20,Make Your Browser Dance,"It was a crisp winter’s evening when I pulled up alongside the pier. I stepped out of my car and the bitterly cold sea air hit my face. I walked around to the boot, opened it and heaved out a heavy flight case. I slammed the boot shut, locked the car and started walking towards the venue.
This was it. My first gig. I thought about all those weeks of preparation: editing video clips, creating 3-D objects, making coloured patterns, then importing them all into software and configuring effects to change as the music did; targeting frequency, beat, velocity, modifying size, colour, starting point; creating playlists of these… and working out ways to mix them as the music played.
This was it. This was me VJing.
This was all a lifetime (well a decade!) ago.
When I started web designing, VJing took a back seat. I was more interested in interactive layouts, semantic accessible HTML, learning all the IE bugs and mastering the quirks that CSS has to offer. More recently, I have been excited by background gradients, 3-D transforms, the @keyframe directive, as well as new APIs such as getUserMedia, indexedDB, the Web Audio API
But wait, have I just come full circle? Could it be possible, with these wonderful new things in technologies I am already familiar with, that I could VJ again, right here, in a browser?
Well, there’s only one thing to do: let’s try it!
Let’s take to the dance floor
Over the past couple of years working in The Lab I have learned to take a much more iterative approach to projects than before. One of my new favourite methods of working is to create a proof of concept to make sure my theory is feasible, before going on to create a full-blown product. So let’s take the same approach here.
The main VJing functionality I want to recreate is manipulating visuals in relation to sound. So for my POC I need to create a visual, with parameters that can be changed, then get some sound and see if I can analyse that sound to detect some data, which I can then use to manipulate the visual parameters. Easy, right?
So, let’s start at the beginning: creating a simple visual. For this I’m going to create a CSS animation. It’s just a funky i element with the opacity being changed to make it flash.
See the Pen Creating a light by Rumyra (@Rumyra) on CodePen
A note about prefixes: I’ve left them out of the code examples in this post to make them easier to read. Please be aware that you may need them. I find a great resource to find out if you do is caniuse.com. You can also check out all the code for the examples in this article
Start the music
Well, that’s pretty easy so far. Next up: loading in some sound. For this we’ll use the Web Audio API. The Web Audio API is based around the concept of nodes. You have a source node: the sound you are loading in; a destination node: usually the device’s speakers; and any number of processing nodes in between. All this processing that goes on with the audio is sandboxed within the AudioContext.
So, let’s start by initialising our audio context.
var contextClass = window.AudioContext;
if (contextClass) {
//web audio api available.
var audioContext = new contextClass();
} else {
//web audio api unavailable
//warn user to upgrade/change browser
}
Now let’s load our sound file into the new context we created with an XMLHttpRequest.
function loadSound() {
//set audio file url
var audioFileUrl = '/octave.ogg';
//create new request
var request = new XMLHttpRequest();
request.open(""GET"", audioFileUrl, true);
request.responseType = ""arraybuffer"";
request.onload = function() {
//take from http request and decode into buffer
context.decodeAudioData(request.response, function(buffer) {
audioBuffer = buffer;
});
}
request.send();
}
Phew! Now we’ve loaded in some sound! There are plenty of things we can do with the Web Audio API: increase volume; add filters; spatialisation. If you want to dig deeper, the O’Reilly Web Audio API book by Boris Smus is available to read online free.
All we really want to do for this proof of concept, however, is analyse the sound data. To do this we really need to know what data we have.
Learning the steps
Let’s take a minute to step back and remember our school days and science class. I’m sure if I drew a picture of a sound wave, we would all start nodding our heads.
The sound you hear is caused by pressure differences in the particles in the air. Sound pushes these particles together, causing vibrations. Amplitude is basically strength of pressure. A simple example of change of amplitude is when you increase the volume on your stereo and the output wave increases in size.
This is great when everything is analogue, but the waveform varies continuously and it’s not suitable for digital processing: there’s an infinite set of values. For digital processing, we need discrete numbers.
We have to sample the waveform at set time intervals, and record data such as amplitude and frequency. Luckily for us, just the fact we have a digital sound file means all this hard work is done for us. What we’re doing in the code above is piping that data in the audio context. All we need to do now is access it.
We can do this with the Web Audio API’s analysing functionality. Just pop in an analysing node before we connect the source to its destination node.
function createAnalyser(source) {
//create analyser node
analyser = audioContext.createAnalyser();
//connect to source
source.connect(analyzer);
//pipe to speakers
analyser.connect(audioContext.destination);
}
The data I’m really interested in here is frequency. Later we could look into amplitude or time, but for now I’m going to stick with frequency.
The analyser node gives us frequency data via the getFrequencyByteData method.
Don’t forget to count!
To collect the data from the getFrequencyByteData method, we need to pass in an empty array (a JavaScript typed array is ideal). But how do we know how many items the array will need when we create it?
This is really up to us and how high the resolution of frequencies we want to analyse is. Remember we talked about sampling the waveform; this happens at a certain rate (sample rate) which you can find out via the audio context’s sampleRate attribute. This is good to bear in mind when you’re thinking about your resolution of frequencies.
var sampleRate = audioContext.sampleRate;
Let’s say your file sample rate is 48,000, making the maximum frequency in the file 24,000Hz (thanks to a wonderful theorem from Dr Harry Nyquist, the maximum frequency in the file is always half the sample rate). The analyser array we’re creating will contain frequencies up to this point. This is ideal as the human ear hears the range 0–20,000hz.
So, if we create an array which has 2,400 items, each frequency recorded will be 10Hz apart. However, we are going to create an array which is half the size of the FFT (fast Fourier transform), which in this case is 2,048 which is the default. You can set it via the fftSize property.
//set our FFT size
analyzer.fftSize = 2048;
//create an empty array with 1024 items
var frequencyData = new Uint8Array(1024);
So, with an array of 1,024 items, and a frequency range of 24,000Hz, we know each item is 24,000 ÷ 1,024 = 23.44Hz apart.
The thing is, we also want that array to be updated constantly. We could use the setInterval or setTimeout methods for this; however, I prefer the new and shiny requestAnimationFrame.
function update() {
//constantly getting feedback from data
requestAnimationFrame(update);
analyzer.getByteFrequencyData(frequencyData);
}
Putting it all together
Sweet sticks! Now we have an array of frequencies from the sound we loaded, updating as the sound plays. Now we want that data to trigger our animation from earlier.
We can easily pause and run our CSS animation from JavaScript:
element.style.webkitAnimationPlayState = ""paused"";
element.style.webkitAnimationPlayState = ""running"";
Unfortunately, this may not be ideal as our animation might be a whole heap longer than just a flashing light. We may want to target specific points within that animation to have it stop and start in a visually pleasing way and perhaps not smack bang in the middle.
There is no really easy way to do this at the moment as Zach Saucier explains in this wonderful article. It takes some jiggery pokery with setInterval to try to ascertain how far through the CSS animation you are in percentage terms.
This seems a bit much for our proof of concept, so let’s backtrack a little. We know by the animation we’ve created which CSS properties we want to change. This is pretty easy to do directly with JavaScript.
element.style.opacity = ""1"";
element.style.opacity = ""0.2"";
So let’s start putting it all together. For this example I want to trigger each light as a different frequency plays. For this, I’ll loop through the HTML elements and change the opacity style if the frequency gain goes over a certain threshold.
//get light elements
var lights = document.getElementsByTagName('i');
var totalLights = lights.length;
for (var i=0; i 160){
//start animation on element
lights[i].style.opacity = ""1"";
} else {
lights[i].style.opacity = ""0.2"";
}
}
See all the code in action here. I suggest viewing in a modern browser :)
Awesome! It is true — we can VJ in our browser!
Let’s dance!
So, let’s start to expand this simple example. First, I feel the need to make lots of lights, rather than just a few. Also, maybe we should try a sound file more suited to gigs or clubs.
Check it out!
I don’t know about you, but I’m pretty excited — that’s just a bit of HTML, CSS and JavaScript!
The other thing to think about, of course, is the sound that you would get at a venue. We don’t want to load sound from a file, but rather pick up on what is playing in real time. The easiest way to do this, I’ve found, is to capture what my laptop’s mic is picking up and piping that back into the audio context. We can do this by using getUserMedia.
Let’s include this in this demo. If you make some noise while viewing the demo, the lights will start to flash.
And relax :)
There you have it. Sit back, play some music and enjoy the Winamp like experience in front of you.
So, where do we go from here? I already have a wealth of ideas. We haven’t started with canvas, SVG or the 3-D features of CSS. There are other things we can detect from the audio as well. And yes, OK, it’s questionable whether the browser is the best environment for this. For one, I’m using a whole bunch of nonsensical HTML elements (maybe each animation could be held within a web component in the future). But hey, it’s fun, and it looks cool and sometimes I think it’s OK to just dance.",2013,Ruth John,ruthjohn,2013-12-02T00:00:00+00:00,https://24ways.org/2013/make-your-browser-dance/,code
70,Bringing Your Code to the Streets,"— or How to Be a Street VJ
Our amazing world of web code is escaping out of the browser at an alarming rate and appearing in every aspect of the environment around us. Over the past few years we’ve already seen JavaScript used server-side, hardware coded with JavaScript, a rise of native style and desktop apps created with HTML, CSS and JavaScript, and even virtual reality (VR) is getting its fair share of front-end goodness.
You can go ahead and play with JavaScript-powered hardware such as the Tessel or the Espruino to name a couple. Just check out the Tessel project page to see JavaScript in the world of coffee roasting or sleep tracking your pet. With the rise of the internet of things, JavaScript can be seen collecting information on flooding among other things. And if that’s not enough ‘outside the browser’ implementations, Node.js servers can even be found in aircraft!
I previously mentioned VR and with three.js’s extra StereoEffect.js module it’s relatively simple to get browser 3D goodness to be Google Cardboard-ready, and thus set the stage for all things JavaScript and VR. It’s been pretty popular in the art world too, with interactive works such as Seb Lee-Delisle’s Lunar Trails installation, featuring the old arcade game Lunar Lander, which you can now play in your browser while others watch (it is the web after all). The Science Museum in London held Chrome Web Lab, an interactive exhibition featuring five experiments, showcasing the magic of the web. And it’s not even the connectivity of the web that’s being showcased; we can even take things offline and use web code for amazing things, such as fighting Ebola.
One thing is for sure, JavaScript is awesome. Hell, if you believe those telly programs (as we all do), JavaScript can even take down the stock market, purely through the witchcraft of canvas! Go JavaScript!
Now it’s our turn
So I wanted to create a little project influenced by this theme, and as it’s Christmas, take it to the streets for a little bit of party fun! Something that could take code anywhere. Here’s how I made a portable visual projection pack, a piece of video mixing software and created some web-coded street art.
Step one: The equipment
You will need:
One laptop: with HDMI output and a modern browser installed, such as Google Chrome.
One battery-powered mini projector: I’ve used a Texas Instruments DLP; for its 120 lumens it was the best cost-to-lumens ratio I could find.
One MIDI controller (optional): mine is an ICON iDJ as it suits mixing visuals. However, there is more affordable hardware on the market such as an Akai LPD8 or a Korg nanoPAD2. As you’ll see in the article, this is optional as it can be emulated within the software.
A case to carry it all around in.
Step two: The software
The projected visuals, I imagined, could be anything you can create within a browser, whether that be simple HTML and CSS, images, videos, SVG or canvas. The only requirement I have is that they move or change with sound and that I can mix any one visual into another.
You may remember a couple of years ago I created a demo on this very site, allowing audio-triggered visuals from the ambient sounds your device mic was picking up. That was a great starting point – I used that exact method to pick up the audio and thus the first requirement was complete. If you want to see some more examples of visuals I’ve put together for this, there’s a showcase on CodePen.
The second requirement took a little more thought. I needed two screens, which could at any point show any of the visuals I had coded, but could be mixed from one into the other and back again. So let’s start with two divs, both absolutely positioned so they’re on top of each other, but at the start the second screen’s opacity is set to zero.
Now all we need is a slider, which when moved from one side to the other slowly sets the second screen’s opacity to 1, thereby fading it in.
See the Pen Mixing Screens (Software Version) by Rumyra (@Rumyra) on CodePen.
Mixing Screens (CodePen)
As you saw above, I have a MIDI controller and although the software method works great, I’d quite like to make use of this nifty piece of kit. That’s easily done with the Web MIDI API. All I need to do is call it, and when I move one of the sliders on the controller (I’ve allocated the big cross fader in the middle for this), pick up on the change of value and use that to control the opacity instead.
var midi, data;
// start talking to MIDI controller
if (navigator.requestMIDIAccess) {
navigator.requestMIDIAccess({
sysex: false
}).then(onMIDISuccess, onMIDIFailure);
} else {
alert(“No MIDI support in your browser.”);
}
// on success
function onMIDISuccess(midiData) {
// this is all our MIDI data
midi = midiData;
var allInputs = midi.allInputs.values();
// loop over all available inputs and listen for any MIDI input
for (var input = allInputs.next(); input && !input.done; input = allInputs.next()) {
// when a MIDI value is received call the onMIDIMessage function
input.value.onmidimessage = onMIDIMessage;
}
}
function onMIDIMessage(message) {
// data comes in the form [command/channel, note, velocity]
data = message.data;
// Opacity change for screen. The cross fader values are [176, 8, {0-127}]
if ( (data[0] === 176) && (data[1] === 8) ) {
// this value will change as the fader is moved
var opacity = data[2]/127;
screenTwo.style.opacity = opacity;
}
}
The final code was slightly more complicated than this, as I decided to switch the two screens based on the frequencies of the sound that was playing, and use the cross fader to depict the frequency threshold value. This meant they flickered in and out of each other, rather than just faded. There’s a very rough-and-ready first version of the software on GitHub.
Phew, Great! Now we need to get all this to the streets!
Step three: Portable kit
Did you notice how I mentioned a case to carry it all around in? I wanted the case to be morphable, so I could use the equipment from it too, a sort of bag-to-usherette-tray-type affair. Well, I had an unused laptop bag…
I strengthened it with some MDF, so when I opened the bag it would hold like a tray where the laptop and MIDI controller would sit. The projector was Velcroed to the external pocket of the bag, so when it was a tray it would project from underneath. I added two durable straps, one for my shoulders and one round my waist, both attached to the bag itself. There was a lot of cutting and trimming. As it was a laptop bag it was pretty thick to start and sewing was tricky. However, I only broke one sewing machine needle; I’ve been known to break more working with leather, so I figured I was doing well. By the way, you can actually buy usherette trays, but I just couldn’t resist hacking my own :)
Step four: Take to the streets
First, make sure everything is charged – everything – a lot! The laptop has to power both the MIDI controller and the projector, and although I have a mobile phone battery booster pack, that’ll only charge the projector should it run out. I estimated I could get a good hour of visual artistry before I needed to worry, though.
I had a couple of ideas about time of day and location. Here in the UK at this time of year, it gets dark around half past four, so I could easily head out in a city around 5pm and it would be dark enough for the projections to be seen pretty well. I chose Bristol, around the waterfront, as there were some interesting locations to try it out in. The best was Millennium Square: busy but not crowded and plenty of surfaces to try projecting on to.
My first time out with the portable audio/visual pack (PAVP as it will now be named) was brilliant. I played music and projected visuals, like a one-woman band of A/V!
You might be thinking what the point of this was, besides, of course, it being a bit of fun. Well, this project got me to look at canvas and SVG more closely. The Web MIDI API was really interesting; MIDI as a data format has some great practical uses. I think without our side projects we may not have all these wonderful uses for our everyday code. Not only do they remind us coding can, and should, be fun, they also help us learn and grow as makers.
My favourite part? When I was projecting into a water feature in Millennium Square. For those who are familiar, you’ll know it’s like a wall of water so it produced a superb effect. I drew quite a crowd and a kid came to stand next to me and all I could hear him say with enthusiasm was, ‘Oh wow! That’s so cool!’
Yes… yes, kid, it was cool. Making things with code is cool.
Massive thanks to the lovely Drew McLellan for his incredibly well-directed photography, and also Simon Johnson who took a great hand in perfecting the kit while it was attached.",2015,Ruth John,ruthjohn,2015-12-06T00:00:00+00:00,https://24ways.org/2015/bringing-your-code-to-the-streets/,code
248,How to Use Audio on the Web,"I know what you’re thinking. I never never want to hear sound anywhere near a browser, ever ever, wow! 🙉
You’re having flashbacks, flashbacks to the days of yore, when we had a element and yup did everyone think that was the most rad thing since . I mean put those two together with a , only use CSS colour names, make sure your borders were all set to ridge and you’ve got yourself the neatest website since 1998.
The sound played when the website loaded and you could play a MIDI file as well! Everyone could hear that wicked digital track you chose. Oh, surfing was gnarly back then.
Yes it is 2018, the end of in fact, soon to be 2019. We are certainly living in the future. Hoverboards self driving cars, holodecks VR headsets, rocket boots drone racing, sound on websites get real, Ruth.
We can’t help but be jaded, even though the element is depreciated, and the autoplay policy appeared this year. Although still in it’s infancy, the policy “controls when video and audio is allowed to autoplay”, which should reduce the somewhat obtrusive playing of sound when a website or app loads in the future.
But then of course comes the question, having lived in a muted present for so long, where and why would you use audio?
✨ Showcase Time ✨
There are some incredible uses of audio on websites today. This is my personal favourite futurelibrary.no, a site from Norway chronicling books that have been published from a forest of trees planted precisely for the books themselves. The sound effects are lovely, adding to the overall experience.
futurelibrary.no
Another site that executes this well is pottermore.com. The Hogwarts WebGL simulation uses both sound effects and ambient background music and gives a great experience. The button hovers are particularly good.
pottermore.com
Eighty-six and a half years is a beautiful narrative site, documenting the musings of an eighty-six and a half year old man. The background music playing on this site is not offensive, it adds to the experience.
Eighty-six and a half years
Sound can be powerful and in some cases useful. Last year I wrote about using them to help validate forms. Audiochart is a library which “allows the user to explore charts on web pages using sound and the keyboard”. Ben Byford recorded voice descriptions of the pages on his website for playback should you need or want it. There is a whole area of accessibility to be explored here.
Then there’s education. Fancy beginning with some piano in the new year? flowkey.com is a website which allows you to play along and learn at the same time. Need to brush up on your music theory? lightnote.co takes you through lessons to do just that, all audio enhanced. Electronic music more your thing? Ableton has your back with learningmusic.ableton.com, a site which takes you through the process of composing electronic music. A website, all made possible through the powers with have with the Web Audio API today.
lightnote.co
learningmusic.ableton.com
Considerations
Yes, tis the season, let’s be more thoughtful about our audios. There are some user experience patterns to begin with. 86andahalfyears.com tells the user they are about to ‘enter’ the site and headphones are recommended. This is a good approach because it a) deals with the autoplay policy (audio needs to be instigated by a user gesture) and b) by stating headphones are recommended you are setting the users expectations, they will expect sound, and if in a public setting can enlist the use of a common electronic device to cause less embarrassment.
Eighty-six and a half years
Allowing mute and/or volume control clearly within the user interface is a good idea. It won’t draw the user out of the experience, it’ll give more control to the user about what audio they want to hear (they may not want to turn down the volume of their entire device), and it’s less thought to reach for a very visible volume than to fumble with device settings.
Indicating that sound is playing is also something to consider. Browsers do this by adding icons to tabs, but this isn’t always the first place to look for everyone.
To The Future
So let’s go!
We see amazing demos built with Web Audio, and I’m sure, like me, they make you think, oh wow I wish I could do that / had thought of that / knew the first thing about audio to begin to even conceive that.
But audio doesn’t actually need to be all bells and whistles (hey, it’s Christmas). Starting, stopping and adjusting simple panning and volume might be all you need to get started to introduce some good sound design in your web design.
Isn’t it great then that there’s a tutorial just for that! Head on over to the MDN Web Audio API docs where the Using the Web Audio API article takes you through playing and pausing sounds, volume control and simple panning (moving the sound from left to right on stereo speakers).
This year I believe we have all experienced the web as a shopping mall more than ever. It’s shining store fronts, flashing adverts, fast food, loud noises.
Let’s use 2019 to create more forests to explore, oceans to dive and mountains to climb.",2018,Ruth John,ruthjohn,2018-12-22T00:00:00+00:00,https://24ways.org/2018/how-to-use-audio-on-the-web/,design
190,Self-Testing Pages with JavaScript,"Working at an agency I am involved more and more on projects in which client side code is developed internally then sent out to a separate team for implementation. You provide static HTML, CSS and JavaScript which then get placed into the CMS and brought to life as an actual website. As you can imagine this can sometimes lead to frustrations. However many safeguards you include, handing over your code to someone else is always a difficult thing to do effectively.
In this article I will show you how you can create a JavaScript implementation checker and that will give you more time for drink based activity as your web site and apps are launched quicker and with less unwanted drama!
An all too frequent occurrence
You’ve been working on a project for weeks, fixed all your bugs and send it to be implemented. You hear nothing and assume all is going well then a few days before it’s meant to launch you get an email from the implementation team informing you of bugs in your code that you need to urgently fix.
The 24ways website with a misspelt ID for the years menu
Being paranoid you trawl through the preview URL, check they have the latest files, check your code for errors then notice that a required HTML attribute has been omitted from the build and therefore CSS or JavaScript you’ve hooked onto that particular attribute isn’t being applied and that’s what is causing the “bug”.
It takes you seconds drafting an email informing them of this, it takes then seconds putting the required attribute in and low and behold the bug is fixed, everyone is happy but you’ve lost a good few hours of your life – this time could have been better spent in the pub.
I’m going to show you a way that these kind of errors can be alerted immediately during implementation of your code and ensure that when you are contacted you know that there actually is a bug to fix. You probably already know the things that could be omitted from a build and look like bugs so you’ll soon be creating tests to look for these and alert when they are not found on the rendered page. The error is reported directly to those who need to know about it and fix it. Less errant bug reports and less frantic emails ahoy!
A page with an implementation issue and instant feedback on the problem
JavaScript selector engines to the rescue
Whether you’re using a library or indeed tapping into the loveliness of the new JavaScript Selector APIs looking for particular HTML elements in JavaScript is fairly trivial now.
For instance this is how you look for a div element with the id attribute of year (the missing attribute from top image) using jQuery (the library I’ll be coding my examples in):
if ($(‘div#year’).length) {
alert(‘win’);
}
Using this logic you can probably imagine how you can write up a quick method to check for the existence of a particular element and alert when it’s not present — but assuming you have a complex page you’re going to be repeating yourself a fair bit and we don’t want to be doing that.
Test scripts
If you’ve got a lot of complex HTML patterns that need testing across a number of different pages it makes sense to keep your tests out of production code. Chances are you’ve already got a load of heavy JavaScript assets, and when it comes to file size saving every little helps.
I don’t think that tests should contain code inside of them so keep mine externally as JSON. This also means that you can use the one set of tests in multiple places. We already know that it’s a good idea to keep our CSS and JavaScript separate so lets continue along those lines here.
The test script for this example looks like this:
{
""title"": ""JS tabs implementation test"",
""description"": ""Check that the correct HTML patterns has been used"",
""author"": ""Ross Bruniges"",
""created"": ""20th July 2009"",
""tests"": [
{
""name"": ""JS tabs elements"",
""description"": ""Checking that correct HTML elements including class/IDs are used on the page for the JS to progressively enhance"",
""selector"": ""div.tabbed_content"",
""message"": ""We couldn't find VAR on the page - it's required for our JavaScript to function correctly"",
""check_for"": {
""contains"": {
""elements"": [
""div.tab_content"", ""h2""
],
""message"": ""We've noticed some missing HTML:please refer to the examples sent for reference""
}
}
}
]
}
The first four lines are just a little bit of meta data so we remember what this test was all about when we look at it again in the future, or indeed if it ever breaks. The tests are the really cool parts and firstly you’ll notice that it’s an array – we’re only going to show one example test here but there is no reason why you can’t place in as many as you want. I’ll explain what each of the lines in the example test means:
name – short test name, I use this in pass/fail messaging later
description – meta data for future reference
selector – the root HTML element from which your HTML will be searched
message – what the app will alert if the initial selector isn’t found
check_for – a wrapper to hold inner tests – those run if the initial selector does match
contains – the type of check, we’re checking that the selector contains specified elements
elements – the HTML elements we are searching for
message – a message for when these don’t match (VAR is substituted when it’s appended to the page with the name of any elements that don’t exist)
It’s very important to pass the function valid JSON (JSONLint is a great tool for this) otherwise you might get a console showing no tests have even been run.
The JavaScript that makes this helpful
Again, this code should never hit a production server so I’ve kept it external. This also means that the only thing that’s needed to be done by the implementation team when they are ready to build is that they delete this code.
“View the full JavaScript:/examples/self-testing-pages-with-javascript/js/tests/test_suite.js
The init function appends the test console to the page and inserts the CSS file required to style it (you don’t need to use pictures of me when tests pass and fail though I see no reason why you shouldn’t), goes and grabs the JSON file referenced and parses it. The methods to pass (tests_pass) and fail (haz_fail) the test I hope are pretty self-explanatory as is the one which creates the test summary once everything has been run (create_summary).
The two interesting functions are init_tests and confirm_html.
init_tests
init_tests:function(i,obj) {
var $master_elm = $(obj.selector);
sleuth.test_page.$logger.append(""
"");
var $container = $('#test_' + i);
if (!$master_elm.length) {
var err_sum = obj.message.replace(/VAR/gi, obj.selector);
sleuth.test_page.haz_failed(err_sum, $container);
return;
}
if (obj.check_for) {
$.each(obj.check_for,function(key, value){
sleuth.test_page.assign_checks($master_elm, $container, key, value);
});
} else {
sleuth.test_page.tests_passed($container);
return;
}
}
The function gets sent the number of the current iteration (used to create a unique id for its test summary) and the current object that contains the data we’re testing against as parameters.
We grab a reference to the root element and this is used (pretty much in the example shown right at the start of this article) and its length is checked. If the length is positive we know we can continue to the inner tests (if they exist) but if not we fail the test and don’t go any further. We append the error to the test console for everyone to see.
If we pass the initial check we send the reference to the root element, message contains and the inner object to a function that in this example sends us on to confirm_html (if we had a more complex test suite it would do a lot more).
confirm_html
confirm_html:function(target_selector, error_elm, obj) {
var missing_elms = [];
$.each(obj.elements, function(i, val) {
if (!target_selector.find(val).length) {
missing_elms.push(val);
}
});
if (missing_elms.length) {
var file_list = missing_elms.join(' ');
var err_sum = obj.message.replace(/VAR/gi, file_list);
sleuth.test_page.haz_failed(err_sum, error_elm);
return;
}
sleuth.test_page.tests_passed(error_elm);
return;
}
We’re again using an array to check for a passed or failed test and checking its length but this time we push in a reference to each missing element we find.
If the test does fail we’re providing even more useful feedback by informing what elements have been missed out. All the implementation team need do is look for them in the files we’ve sent and include them as expected.
No more silly implementation bugs!
Here is an example of a successful implementation.
Here are some examples of failed implementations – one which fails at finding the root node and one that has the correct root node but none of the inner HTML tests pass.
Is this all we can check for?
Certainly not!
JavaScript provides pretty easy ways to check for attributes, included files (if the files being checked for are being referenced correctly and not 404ing) and even applied CSS.
Want to check that those ARIA attributes are being implemented correctly or that all images contain an alt attribute well this simple test suite can be extended to include tests for this – the sky is pretty much up to your imagination.",2009,Ross Bruniges,rossbruniges,2009-12-12T00:00:00+00:00,https://24ways.org/2009/self-testing-pages-with-javascript/,process
63,Be Fluid with Your Design Skills: Build Your Own Sites,"Just five years ago in 2010, when we were all busy trying to surprise and delight, learning CSS3 and trying to get whole websites onto one page, we had a poster on our studio wall. It was entitled ‘Designers Vs Developers’, an infographic that showed us the differences between the men(!) who created websites.
Designers wore skinny jeans and used Macs and developers wore cargo pants and brought their own keyboards to work. We began to learn that designers and developers were not only doing completely different jobs but were completely different people in every way. This opinion was backed up by hundreds of memes, millions of tweets and pages of articles which used words like void and battle and versus.
Thankfully, things move quickly in this industry; the wide world of web design has moved on in the last five years. There are new devices, technologies, tools – and even a few women. Designers have been helped along by great apps, software, open source projects, conferences, and a community of people who, to my unending pride, love to share their knowledge and their work.
So the world has moved on, and if Miley Cyrus, Ruby Rose and Eliot Sumner are identifying as gender fluid (an identity which refers to a gender which varies over time or is a combination of identities), then I would like to come out as discipline fluid!
OK, I will probably never identify as a developer, but I will identify as fluid! How can we be anything else in an industry that moves so quickly? That’s how we should think of our skills, our interests and even our job titles. After all, Steve Jobs told us that “Design is not just what it looks like and feels like. Design is how it works.” Sorry skinny-jean-wearing designers – this means we’re all designing something together. And it’s not just about knowing the right words to use: you have to know how it feels. How it feels when you make something work, when you fix that bug, when you make it work on IE.
Like anything in life, things run smoothly when you make the effort to share experiences, empathise and deeply understand the needs of others. How can designers do that if they’ve never built their own site? I’m not talking the big stuff, I’m talking about your portfolio site, your mate’s business website, a website for that great idea you’ve had. I’m talking about doing it yourself to get an unique insight into how it feels.
We all know that designers and developers alike love an , so here it is.
Ten reasons designers should be fluid with their skills and build their own sites
1. It’s never been easier
Now here’s where the definition of ‘build’ is going to get a bit loose and people are going to get angry, but when I say it’s never been easier I mean because of the existence of apps and software like WordPress, Squarespace, Tumblr, et al. It’s easy to make something and get it out there into the world, and these are all gateway drugs to hard coding!
2. You’ll understand how it feels
How it feels to be so proud that something actually works that you momentarily don’t notice if the kerning is off or the padding is inconsistent. How it feels to see your site appear when you’ve redirected a URL. How it feels when you just can’t work out where that one extra space is in a line of PHP that has killed your whole site.
3. It makes you a designer
Not a better designer, it makes you a designer when you are designing how things look and how they work.
4. You learn about movement
Photoshop and Sketch just don’t cut it yet. Until you see your site in a browser or your app on a phone, it’s hard to imagine how it moves. Building your own sites shows you that it’s not just about how the content looks on the screen, but how it moves, interacts and feels.
5. You make techie friends
All the tutorials and forums in the world can’t beat your network of techie friends. Since I started working in web design I have worked with, sat next to, and co-created with some of the greatest developers. Developers who’ve shared their knowledge, encouraged me to build things, patiently explained HTML, CSS, servers, divs, web fonts, iOS development. There has been no void, no versus, very few battles; just people who share an interest and love of making things.
6. You will own domain names
When something is paid for, online and searchable then it’s real and you’ve got to put the work in. Buying domains has taught me how to stop procrastinating, but also about DNS, FTP, email, and how servers work.
7. People will ask you to do things
Learning about code and development opens a whole new world of design. When you put your own personal websites and projects out there people ask you to do more things. OK, so sometimes those things are “Make me a website for free”, but more often it’s cool things like “Come and speak at my conference”, “Write an article for my magazine” and “Collaborate with me.”
8. The young people are coming!
They love typography, they love print, they love layout, but they’ve known how to put a website together since they started their first blog aged five and they show me clever apps they’ve knocked together over the weekend! They’re new, they’re fluid, and they’re better than us!
9. Your portfolio is your portfolio
OK, it’s an obvious one, but as designers our work is our CV, our legacy! We need to show our skill, our attention to detail and our creativity in the way we showcase our work. Building your portfolio is the best way to start building your own websites. (And please be that designer who’s bothered to work out how to change the Squarespace favicon!)
10. It keeps you fluid!
Building your own websites is tough. You’ll never be happy with it, you’ll constantly be updating it to keep up with technology and fashion, and by the time you’ve finished it you’ll want to start all over again. Perfect for forcing you to stay up-to-date with what’s going on in the industry.
",2015,Ros Horner,roshorner,2015-12-12T00:00:00+00:00,https://24ways.org/2015/be-fluid-with-your-design-skills-build-your-own-sites/,code
334,Transitional vs. Strict Markup,"When promoting web standards, standardistas often talk about XHTML as being more strict than HTML. In a sense it is, since it requires that all elements are properly closed and that attribute values are quoted. But there are two flavours of XHTML 1.0 (three if you count the Frameset DOCTYPE, which is outside the scope of this article), defined by the Transitional and Strict DOCTYPEs. And HTML 4.01 also comes in those flavours.
The names reveal what they are about: Transitional DOCTYPEs are meant for those making the transition from older markup to modern ways. Strict DOCTYPEs are actually the default – the way HTML 4.01 and XHTML 1.0 were constructed to be used.
A Transitional DOCTYPE may be used when you have a lot of legacy markup that cannot easily be converted to comply with a Strict DOCTYPE. But Strict is what you should be aiming for. It encourages, and in some cases enforces, the separation of structure and presentation, moving the presentational aspects from markup to CSS. From the HTML 4 Document Type Definition:
This is HTML 4.01 Strict DTD, which excludes the presentation attributes and elements that W3C expects to phase out as support for style sheets matures. Authors should use the Strict DTD when possible, but may use the Transitional DTD when support for presentation attribute and elements is required.
An additional benefit of using a Strict DOCTYPE is that doing so will ensure that browsers use their strictest, most standards compliant rendering modes.
Tommy Olsson provides a good summary of the benefits of using Strict over Transitional in Ten questions for Tommy Olsson at Web Standards Group:
In my opinion, using a Strict DTD, either HTML 4.01 Strict or XHTML 1.0 Strict, is far more important for the quality of the future web than whether or not there is an X in front of the name. The Strict DTD promotes a separation of structure and presentation, which makes a site so much easier to maintain.
For those looking to start using web standards and valid, semantic markup, it is important to understand the difference between Transitional and Strict DOCTYPEs. For complete listings of the differences between Transitional and Strict DOCTYPEs, see XHTML: Differences between Strict & Transitional, Comparison of Strict and Transitional XHTML, and XHTML1.0 Element Attributes by DTD.
Some of the differences are more likely than others to cause problems for developers moving from a Transitional DOCTYPE to a Strict one, and I’d like to mention a few of those.
Elements that are not allowed in Strict DOCTYPEs
center
font
iframe
strike
u
Attributes not allowed in Strict DOCTYPEs
align (allowed on elements related to tables: col, colgroup, tbody, td, tfoot, th, thead, and tr)
language
background
bgcolor
border (allowed on table)
height (allowed on img and object)
hspace
name (allowed in HTML 4.01 Strict, not allowed on form and img in XHTML 1.0 Strict)
noshade
nowrap
target
text, link, vlink, and alink
vspace
width (allowed on img, object, table, col, and colgroup)
Content model differences
An element type’s content model describes what may be contained by an instance of the element type. The most important difference in content models between Transitional and Strict is that blockquote, body, and form elements may only contain block level elements. A few examples:
text and images are not allowed immediately inside the body element, and need to be contained in a block level element like p or div
input elements must not be direct descendants of a form element
text in blockquote elements must be wrapped in a block level element like p or div
Go Strict and move all presentation to CSS
Something that can be helpful when doing the transition from Transitional to Strict DOCTYPEs is to focus on what each element of the page you are working on is instead of how you want it to look.
Worry about looks later and get the structure and semantics right first.",2005,Roger Johansson,rogerjohansson,2005-12-13T00:00:00+00:00,https://24ways.org/2005/transitional-vs-strict-markup/,code
279,Design the Invisible to Tell Better Stories on the Web,"For design to be meaningful we need to tell stories. We need to design the invisible, the cues, the messages and the extra detail hidden beneath the aesthetics. It’s all about the story.
From verbal exchanges around the campfire to books, the web and everything in between, storytelling allows us to share, organize and process information more efficiently. It helps us understand our surroundings and make emotional connections to people, places and experiences.
Web design lends itself perfectly to the conventions of storytelling, a universal process. However, the stories vary because they’re defined by culture, society, politics and religion. All of which need considering if you are to design stories that are relevant to your target audience.
The benefits of approaching design with storytelling in mind from the very start of the project is that we are creating considered design that allows users to quickly gather meaning from the website. They do this by reading between the lines and drawing on the wealth of knowledge they have acquired about the associations between colours, typyefaces and signs.
With so much recognition and analysis happening subconsciously you have to consider how design communicates on this level. This invisible layer has a significant impact on what you say, how you say it and who you say it to.
How can we design something that’s invisible?
By researching and making conscious decisions about exactly what you are communicating, you can make the invisible visible. As is often quoted, good design is like air, you only notice it when it’s bad. So by designing the invisible the aim is to design stories that the audience receive subliminally, so that they go somewhat unnoticed, like good air.
Storytelling strands
To share these stories through design, you can break it down into several strands. Each strand tells a story on its own, but when combined they may start to tell a different story altogether. These strands are colour, typefaces, branding, tone of voice and symbols. All are literal and visible but the invisible element is the meaning behind them – meaning that you can extract and share. In this article I want to focus on colour, typefaces and tone of voice and on how combining story strands can change the meaning.
Colour
Let’s start with colour. Red represents emotions such as love but can also signify war. Green is commonly used for all things environmental and purple is a colour that connotes wealth and royalty. These associations between colour and emotion or value have been learned over time and are continually reinforced through media and culture.
With this knowledge come expectations from your users. For example, they will expect Valentine’s Day sites to be red and kids’ sites to be bright and colourful. This is true in the same way audiences have expectations of certain genres of film or music. These conventions help savvy audiences decode texts and read between the lines or, rather, to draw meaning from the invisible. It’s practically an innate skill. This is why you need to design the invisible: because users will quickly deduce meaning from your site and fill in the story’s gaps, it’s important to give them as much of that story to begin with. A story relevant to their culture.
Of all the ways you can tell stories through web design, colour is the most fascinating and important. Not only does it evoke emotions in users but its meaning varies significantly between cultures. In the west, for example, white is a colour associated with weddings, and black is the colour of mourning. This is signified by the traditions of brides wearing white and those in mourning wearing black. In other cultures the meanings are reversed, as black is a colour that represents good luck and white is a colour that signifies mourning. If you assume the same values are true in all cultures then you risk offending the very people you are targeting.
When colours combine, the story being told can change. If you design using red, white and blue then it’s going to be difficult to shake off patriotic connotations because this colour combination is so ingrained as being American or British or French thanks largely to their flags. This extends to politics too. Each party has its own representative colour. In the UK, the Conservatives are blue and Labour is red so it would be inappropriate storytelling to design a Labour-related website in blue as there would be a conflict between the content and the design, a conflict that would result in a poor user experience.
Conflicts become more of an issue when you start to combine story strands. I once saw a No VAT advert use the symbol on the left:
There’s a complete conflict in storytelling here between the sign and its colour. The prohibition sign was used over the word VAT to mean no VAT; that makes sense. But this is a symbol that is used to communicate to people that something is being prohibited or prevented, it mustn’t continue. So to use green contradicts the message of the sign itself; green is used as a colour to say yes, go, proceed, enter. The same would be true if we had a tick in red and a cross in green. Bad design here means the story is flawed and the user experience is compromised.
Typefaces
Typefaces also tell stories. They are so much more than the words that are written with them because they connote different values. Here are a few:
Serif fonts are more formal and are associated with tradition, sophistication and high-end values. Sans serif fonts, on the other hand, are synonymous with modernity, informality and friendliness. These perceptions are again reinforced through more traditional media such as newspaper mastheads, where the serious news-focused broadsheets have serif titles, and the showbiz and gossip-led tabloids have sans serif titles. This translates to the web as well. With these associations already familiar to users, they may see copy and focus on the words, but if the way that copy is displayed jars with the context then we are back to having conflicting stories like the No VAT sign earlier.
Let’s take official institutions, for example. The White House, the monarchy, 10 Downing Street and other government departments are formal, traditional and important organisations. If the copy on their websites were written in a typeface like Cooper Black, it would erase any authority and respect that they were due. They need people to take them seriously and trust them, and part of the way to do this is to have a typeface that represents those values.
It works both ways though. If Innocent, Threadless or other fun companies used traditional typefaces, they wouldn’t be accurate reflections of their core values, brand and personality. They are better positioned to use friendly, informal and modern typefaces. But still never Comic Sans.
Tone of voice
Closely tied to this is tone of voice, my absolute bugbear on the web. Tone of voice isn’t what is said but, rather, how it is said. When we interact with others in person we don’t just listen to the words they say, but we also draw meaning from their body language, and pitch and tone of voice. Just because the web removes that face-to-face interaction with your audience it doesn’t mean you can’t have a tone of voice.
Innocent pioneered the informal chatty tone of voice that so many others have since emulated, but unless it is representative of your company, then it’s not appropriate. It works for Innocent because the tone of voice is consistent across all the company’s materials, both online and offline. Ben and Jerry’s takes the same approach, as does Threadless, but maybe you need a more formal or corporate tone of voice. It really depends on what your business or service is and who it is for, and that is why I think LoveFilm has it all wrong.
LoveFilm offers a film and game rental service, something fun for people in their downtime. While they aren’t particularly stuffy, neither is their tone of voice very friendly or informal, which is what I would expect from a service like theirs. The reason they have it wrong is in the language they use and the way their sentences are constructed.
This is the first time we’ve discussed language because, on the whole, designing the invisible isn’t concerned with language at all. But that doesn’t mean that these strands can’t still elicit an emotional response in users. Jon Tan quoted Dr Mazviita Chirimuuta in his New Adventures in Web Design talk in January 2011:
Although there is no absolute separation between language and emotion, there will still be countless instances where you have emotional response without verbal input or linguistic cognition. In general language is not necessary for emotion.
This is even more pertinent when the emotions evoked are connected to people’s culture, surroundings and way of life. It makes design personal, something that audiences can connect with at more than just face value but, rather, on a subliminal or, indeed, invisible level.
It also means that when you are asked the inevitable question of why – why is blue the dominant colour? why have you used that typeface? why don’t we sound like Innocent? – you will have a rationale behind each design decision that can explain what story you are telling, how you discovered the story and how it is targeted at the core audience.
Research
This is where research plays a vital role in the project cycle. If you don’t know and understand your audience then you don’t know what story to design. Every project lends itself to some level of research, but how in-depth and what methods are most appropriate will be dictated by project requirements and budget restrictions – but do your research.
Even if you think you know your audience, it doesn’t hurt to research and reaffirm that because cultures and society do change, albeit slowly, but they can change. So ask questions at the start of the project during the research phase:
What do different colours mean for your audience’s culture?
Do the typeface and tone of voice appeal to the demographic?
Does the brand identity represent the values and personality of your service?
Are there any social, political or religious significances associated with your audience that you need to take into consideration so you don’t offend them?
Ask questions, understand your audience, design your story based on these insights, and create better user experiences in context that have good, solid storytelling at their heart.
Major hat tip to Gareth Strange for the beautiful graphics used within this article.",2011,Robert Mills,robertmills,2011-12-14T00:00:00+00:00,https://24ways.org/2011/design-the-invisible/,design
53,Get Expressive with Your Typography,"In 1955 Beatrice Warde, an American communicator on typography, published a series of essays entitled The Crystal Goblet in which she wrote, “People who love ideas must have a love of words. They will take a vivid interest in the clothes that words wear.” And with that proposition Warde introduced the idea that just as we judge someone based on the clothes they are wearing, so we make judgements about text based on the typefaces in which it is set.
Beatrice Warde. ©1970 Monotype Imaging Inc.
Choosing the same typeface as everyone else, especially if you’re trying to make a statement, is like turning up to a party in the same dress; to a meeting in the same suit, shirt and tie; or to a craft ale dispensary in the same plaid shirt and turned-up skinny jeans.
But there’s more to your choice of typeface than simply making an impression. In 2012 Jon Tan wrote on 24 ways about a scientific study called “The Aesthetics of Reading” which concluded that “good quality typography is responsible for greater engagement during reading and thus induces a good mood.”
Furthermore, at this year’s Ampersand conference Sarah Hyndman, an expert in multisensory typography, discussed how typefaces can communicate with our subconscious. Sarah showed that different fonts could have an effect on how food tasted. A rounded font placed near a bowl of jellybeans would make them taste sweeter, and a jagged angular font would make them taste more sour.
The quality of your typography can therefore affect the mood of your reader, and your font choice directly affect the senses. This means you can manipulate the way people feel. You can change their emotional state through type alone. Now that’s a real superpower!
The effects of your body text design choices are measurable but subtle. If you really want to have an impact you need to think big. Literally. Display text and headings are your attention grabbers. They are your chance to interrupt, introduce and seduce.
Display text and headings set the scene and draw people in. Text set large creates an image that visitors see before they read, and that’s your chance to choose a typeface that immediately expresses what the text, and indeed the entire website, stands for. What expectations of the text do you want to set up? Youthful enthusiasm? Businesslike? Cutting-edge? Hipster? Sensible and secure? Fun and informal? Authoritarian?
Typography conveys much more than just information. It imparts feeling, emotion and sentiment, and arouses preconceived ideas of trust, tone and content. Think about taking advantage of this by introducing impactful, expressive typography to your designs on the web. You can alter the way your reader feels, so what emotion do you want to provoke?
Maybe you want them to feel inspired like this stop smoking campaign:
helsenorge.no
Perhaps they should be moved and intrigued, as with Makeshift magazine:
mkshft.org
Or calmly reassured:
www.cleopatra-marina.gr
Fonts also tap into the complex library of associations that we’ve been accumulating in our brains all of our lives. You build up these associations every time you see a font from the context that you see it in. All of us associate certain letterforms with topics, times and places.
Retiro is obviously Spanish:
Retiro by Typofonderie
Bodoni and Eurostile used in this menu couldn’t be much more Italian:
Bodoni and Eurostile, both designed in Italy
To me, Clarendon gives a sense of the 1960s and 1970s. I’m not sure if that’s what Costa was going for, but that’s what it means to me:
Costa coffee flier
And Knockout and Gotham really couldn’t be much more American:
Knockout and Gotham by Hoefler & Co
When it comes to choosing your display typeface, the type designer Christian Schwartz says there are two kinds. First are the workhorse typefaces that will do whatever you want them to do. Helvetica, Proxima Nova and Futura are good examples. These fonts can be shaped in many different ways, but this also means they are found everywhere and take great skill and practice to work with in a unique and striking manner.
The second kind of typeface is one that does most of the work for you. Like finely tailored clothing, it’s the detail in the design that adds interest.
Setting headings in Bree rather than Helvetica makes a big difference to the tone of the article
Such typefaces carry much more inherent character, but are also less malleable and harder to adapt to different contexts. Good examples are Marr Sans, FS Clerkenwell, Strangelove and Bree.
Push the boat out
Remember, all type can have an effect on the reader. Take advantage of that and allow your type to have its own vernacular and impact. Be expressive with your type. Don’t be too reverential, dogmatic – or ordinary. Be brave and push a few boundaries.
Adapted from Web Typography a book in progress by Richard Rutter.",2015,Richard Rutter,richardrutter,2015-12-04T00:00:00+00:00,https://24ways.org/2015/get-expressive-with-your-typography/,design
120,Easier Page States for Wireframes,"When designing wireframes for web sites and web apps, it is often overlooked that the same ‘page’ can look wildly different depending on its context. A logged-in page will look different from a logged-out page; an administrator’s view may have different buttons than a regular user’s view; a power user’s profile will be more extensive than a new user’s.
These different page states need designing at some point, especially if the wireframes are to form a useful communication medium between designer and developer. Documenting the different permutations can be a time consuming exercise involving either multiple pages in one’s preferred box-and-arrow software, or a fully fledged drawing containing all the possible combinations annotated accordingly.
Enter interactive wireframes and Polypage
Interactive wireframes built in HTML are a great design and communication tool. They provide a clickable prototype, running in the browser as would the final site. As such they give a great feel for how the site will be to use. Once you add in the possibilities of JavaScript and a library such as jQuery, they become even more flexible and powerful.
Polypage is a jQuery plugin which makes it really easy to design multiple page states in HTML wireframes. There’s no JavaScript knowledge required (other than cutting and pasting in a few lines). The page views are created by simply writing all the alternatives into your HTML page and adding special class names to apply state and conditional view logic to the various options.
When the page is loaded Polypage automatically detects the page states defined by the class names and creates a control bar enabling the user to toggle page states with the click of a mouse or the clack of a keyboard.
Using cookies by way of the jQuery cookie plugin, Polypage retains the view state throughout your prototype. This means you could navigate through your wireframes as if you were logged out; as if you were logged in as an administrator; with notes on or off; or with any other view or state you might require. The possibilities are entirely up to you.
How does it work?
Firstly you need to link to jQuery, the jQuery cookie plugin and to Polypage. Something like this:
Then you need to initialise Polypage on page load using something along these lines:
Next you need to define the areas of your wireframe which are particular to a given state or view. Do this by applying classes beginning with pp_. Polypage will ignore all other classes in the document.
The pp_ prefix should be followed by a state name. This can be any text string you like, bearing in mind it will appear in the control bar. Typical page states might include ‘logged_in’, ‘administrator’ or ‘group_owner’. A complete class name would therefore look something like pp_logged_in.
Examples
If a user is logged in, you might want to specify an option for him or her to sign out. Using Polypage, this could be put in the wireframe as follows:
Sign out
Polypage will identify the pp_logged_in class on the link and hide it (as the ‘Sign out’ link should only be shown when the page is in the ‘logged in’ view). Polypage will then automatically write a ‘logged in’ toggle to the control bar, enabling you to show or hide the ‘Sign out’ link by toggling the ‘logged in’ view. The same will apply to all content marked with a pp_logged_in class.
States can also be negated by adding a not keyword to the class name. For example you might want to provide a log in link for users who are not signed in. Using Polypage, you would insert the not keyword after the pp prefix as follows:
Login
Again Polypage identifies the pp prefix but this time sees that the ‘Login’ link should not be shown when the ‘logged in’ state is selected.
States can also be joined together to add some basic logic to pages. The syntax follows natural language and uses the or and and keywords in addition to the afore-mentioned not. Some examples would be pp_logged_in_and_admin, pp_admin_or_group_owner and pp_logged_in_and_not_admin.
Finally, you can set default states for a page by passing an array to the polypage.init() function like this:
$.polypage.init(['logged_in', 'admin']);
You can see a fully fledged example in this fictional social network group page. The example page defaults to a logged in state. You can see the logged out state by toggling ‘logged in’ off in the Polypage control bar. There are also views specified for a group member, a group admin, a new group and notes.
Where can I get hold of it?
You can download the current version from GitHub.
Polypage was originally developed by Clearleft and New Bamboo, with particular contributions from Andy Kent and Natalie Downe. It has been used in numerous real projects, but it is still an early release so there is bound to be room for improvement. We’re pleased to say that Polypage is now an open source project so any feedback, particularly by way of actual improvements, is extremely welcome.",2008,Richard Rutter,richardrutter,2008-12-11T00:00:00+00:00,https://24ways.org/2008/easier-page-states-for-wireframes/,process
141,Compose to a Vertical Rhythm,"“Space in typography is like time in music. It is infinitely divisible, but a few proportional intervals can be much more useful than a limitless choice of arbitrary quantities.” So says the typographer Robert Bringhurst, and just as regular use of time provides rhythm in music, so regular use of space provides rhythm in typography, and without rhythm the listener, or the reader, becomes disorientated and lost.
On the Web, vertical rhythm – the spacing and arrangement of text as the reader descends the page – is contributed to by three factors: font size, line height and margin or padding. All of these factors must calculated with care in order that the rhythm is maintained.
The basic unit of vertical space is line height. Establishing a suitable line height that can be applied to all text on the page, be it heading, body copy or sidenote, is the key to a solid dependable vertical rhythm, which will engage and guide the reader down the page. To see this in action, I’ve created an example with headings, footnotes and sidenotes.
Establishing a suitable line height
The easiest place to begin determining a basic line height unit is with the font size of the body copy. For the example I’ve chosen 12px. To ensure readability the body text will almost certainly need some leading, that is to say spacing between the lines. A line-height of 1.5em would give 6px spacing between the lines of body copy. This will create a total line height of 18px, which becomes our basic unit. Here’s the CSS to get us to this point:
body {
font-size: 75%;
}
html>body {
font-size: 12px;
}
p {
line-height 1.5em;
}
There are many ways to size text in CSS and the above approach provides and accessible method of achieving the pixel-precision solid typography requires. By way of explanation, the first font-size reduces the body text from the 16px default (common to most browsers and OS set-ups) down to the 12px we require. This rule is primarily there for Internet Explorer 6 and below on Windows: the percentage value means that the text will scale predictably should a user bump the text size up or down. The second font-size sets the text size specifically and is ignored by IE6, but used by Firefox, Safari, IE7, Opera and other modern browsers which allow users to resize text sized in pixels.
Spacing between paragraphs
With our rhythmic unit set at 18px we need to ensure that it is maintained throughout the body copy. A common place to lose the rhythm is the gaps set between margins. The default treatment by web browsers of paragraphs is to insert a top- and bottom-margin of 1em. In our case this would give a spacing between the paragraphs of 12px and hence throw the text out of rhythm. If the rhythm of the page is to be maintained, the spacing of paragraphs should be related to the basic line height unit. This is achieved simply by setting top- and bottom-margins equal to the line height.
In order that typographic integrity is maintained when text is resized by the user we must use ems for all our vertical measurements, including line-height, padding and margins.
p {
font-size:1em;
margin-top: 1.5em;
margin-bottom: 1.5em;
}
Browsers set margins on all block-level elements (such as headings, lists and blockquotes) so a way of ensuring that typographic attention is paid to all such elements is to reset the margins at the beginning of your style sheet. You could use a rule such as:
body,div,dl,dt,dd,ul,ol,li,h1,h2,h3,h4,h5,h6,pre,form,fieldset,p,blockquote,th,td {
margin:0;
padding:0;
}
Alternatively you could look into using the Yahoo! UI Reset style sheet which removes most default styling, so providing a solid foundation upon which you can explicitly declare your design intentions.
Variations in text size
When there is a change in text size, perhaps with a heading or sidenotes, the differing text should also take up a multiple of the basic leading. This means that, in our example, every diversion from the basic text size should take up multiples of 18px. This can be accomplished by adjusting the line-height and margin accordingly, as described following.
Headings
Subheadings in the example page are set to 14px. In order that the height of each line is 18px, the line-height should be set to 18 ÷ 14 = 1.286. Similarly the margins above and below the heading must be adjusted to fit. The temptation is to set heading margins to a simple 1em, but in order to maintain the rhythm, the top and bottom margins should be set at 1.286em so that the spacing is equal to the full 18px unit.
h2 {
font-size:1.1667em;
line-height: 1.286em;
margin-top: 1.286em;
margin-bottom: 1.286em;
}
One can also set asymmetrical margins for headings, provided the margins combine to be multiples of the basic line height. In our example, a top margin of 1½ lines is combined with a bottom margin of half a line as follows:
h2 {
font-size:1.1667em;
line-height: 1.286em;
margin-top: 1.929em;
margin-bottom: 0.643em;
}
Also in our example, the main heading is given a text size of 18px, therefore the line-height has been set to 1em, as has the margin:
h1 {
font-size:1.5em;
line-height: 1em;
margin-top: 0;
margin-bottom: 1em;
}
Sidenotes
Sidenotes (and other supplementary material) are often set at a smaller size to the basic text. To keep the rhythm, this smaller text should still line up with body copy, so a calculation similar to that for headings is required. In our example, the sidenotes are set at 10px and so their line-height must be increased to 18 ÷ 10 = 1.8.
.sidenote {
font-size:0.8333em;
line-height:1.8em;
}
Borders
One additional point where vertical rhythm is often lost is with the introduction of horizontal borders. These effectively act as shims pushing the subsequent text downwards, so a two pixel horizontal border will throw out the vertical rhythm by two pixels. A way around this is to specify horizontal lines using background images or, as in our example, specify the width of the border in ems and adjust the padding to take up the slack.
The design of the footnote in our example requires a 1px horizontal border. The footnote contains 12px text, so 1px in ems is 1 ÷ 12 = 0.0833. I have added a margin of 1½ lines above the border (1.5 × 18 ÷ 12 = 2.5ems), so to maintain the rhythm the border + padding must equal a ½ (9px). We know the border is set to 1px, so the padding must be set to 8px. To specify this in ems we use the familiar calculation: 8 ÷ 12 = 0.667.
Hit me with your rhythm stick
Composing to a vertical rhythm helps engage and guide the reader down the page, but it takes typographic discipline to do so. It may seem like a lot of fiddly maths is involved (a few divisions and multiplications never hurt anyone) but good type setting is all about numbers, and it is this attention to detail which is the key to success.",2006,Richard Rutter,richardrutter,2006-12-12T00:00:00+00:00,https://24ways.org/2006/compose-to-a-vertical-rhythm/,design
146,Increase Your Font Stacks With Font Matrix,"Web pages built in plain old HTML and CSS are displayed using only the fonts installed on users’ computers (@font-face implementations excepted). To enable this, CSS provides the font-family property for specifying fonts in order of preference (often known as a font stack). For example:
h1 {font-family: 'Egyptienne F', Cambria, Georgia, serif}
So in the above rule, headings will be displayed in Egyptienne F. If Egyptienne F is not available then Cambria will be used, failing that Georgia or the final fallback default serif font. This everyday bit of CSS will be common knowledge among all 24 ways readers.
It is also a commonly held belief that the only fonts we can rely on being installed on users’ computers are the core web fonts of Arial, Times New Roman, Verdana, Georgia and friends. But is that really true?
If you look in the fonts folder of your computer, or even your Mum’s computer, then you are likely to find a whole load of fonts besides the core ones. This is because many software packages automatically install extra typefaces. For example, Office 2003 installs over 100 additional fonts. Admittedly not all of these fonts are particularly refined, and not all are suitable for the Web. However they still do increase your options.
The Matrix
I have put together a matrix of (western) fonts showing which are installed with Mac and Windows operating systems, which are installed with various versions of Microsoft Office, and which are installed with Adobe Creative Suite.
The matrix is available for download as an Excel file and as a CSV. There are no readily available statistics regarding the penetration of Office or Creative Suite, but you can probably take an educated guess based on your knowledge of your readers.
The idea of the matrix is that use can use it to help construct your font stack. First of all pick the font you’d really like for your text – this doesn’t have to be in the matrix. Then pick the generic family (serif, sans-serif, cursive, fantasy or monospace) and a font from each of the operating systems. Then pick any suitable fonts from the Office and Creative Suite lists.
For example, you may decide your headings should be in the increasingly ubiquitous Clarendon. This is a serif type face. At OS-level the most similar is arguably Georgia. Adobe CS2 comes with Century Old Style which has a similar feel. Century Schoolbook is similar too, and is installed with all versions of Office. Based on this your font stack becomes:
font-family: 'Clarendon Std', 'Century Old Style Std', 'Century Schoolbook', Georgia, serif
Note the ‘Std’ suffix indicating a ‘standard’ OpenType file, which will normally be your best bet for more esoteric fonts.
I’m not suggesting the process of choosing suitable fonts is an easy one. Firstly there are nearly two hundred fonts in the matrix, so learning what each font looks like is tricky and potentially time consuming (if you haven’t got all the fonts installed on a machine to hand you’ll be doing a lot of Googling for previews). And it’s not just as simple as choosing fonts that look similar or have related typographic backgrounds, they need to have similar metrics as well, This is especially true in terms of x-height which gives an indication of how big or small a font looks.
Over to You
The main point of all this is that there are potentially more fonts to consider than is generally accepted, so branch out a little (carefully and tastefully) and bring a little variety to sites out there. If you come up with any novel font stacks based on this approach, please do blog them (tagged as per the footer) and at some point they could all be combined in one place for everyone to consider.
Appendix
What about Linux?
The only operating systems in the matrix are those from Microsoft and Apple. For completeness, Linux operating systems should be included too, although these are many and varied and very much in a minority, so I omitted them for time being. For the record, some Linux distributions come packaged with Microsoft’s core fonts. Others use the Vera family, and others use the Liberation family which comprises fonts metrically identical to Times New Roman and Arial.
Sources
The sources of font information for the matrix are as follows:
Windows XP SP2
Windows Vista
Office 2003
Office 2007
Mac OSX Tiger
Mac OSX Leopard (scroll down two thirds)
Office 2004 (Mac) by inspecting my Microsoft Office 2004/Office/Fonts folder
Office 2008 (Mac) is expected to be as Office 2004 with the addition of the Vista ClearType fonts
Creative Suite 2 (see pdf link in first comment)
Creative Suite 3",2007,Richard Rutter,richardrutter,2007-12-17T00:00:00+00:00,https://24ways.org/2007/increase-your-font-stacks-with-font-matrix/,design
239,Using the WebFont Loader to Make Browsers Behave the Same,"Web fonts give us designers a whole new typographic palette with which to work. However, browsers handle the loading of web fonts in different ways, and this can lead to inconsistent user experiences.
Safari, Chrome and Internet Explorer leave a blank space in place of the styled text while the web font is loading. Opera and Firefox show text with the default font which switches over when the web font has loaded, resulting in the so-called Flash of Unstyled Text (aka FOUT). Some people prefer Safari’s approach as it eliminates FOUT, others think the Firefox way is more appropriate as content can be read whilst fonts download. Whatever your preference, the WebFont Loader can make all browsers behave the same way.
The WebFont Loader is a JavaScript library that gives you extra control over font loading. It was co-developed by Google and Typekit, and released as open source. The WebFont Loader works with most web font services as well as with self-hosted fonts.
The WebFont Loader tells you when the following events happen as a browser downloads web fonts (or loads them from cache):
when fonts start to download (‘loading’)
when fonts finish loading (‘active’)
if fonts fail to load (‘inactive’)
If your web page requires more than one font, the WebFont Loader will trigger events for individual fonts, and for all the fonts as a whole. This means you can find out when any single font has loaded, and when all the fonts have loaded (or failed to do so).
The WebFont Loader notifies you of these events in two ways: by applying special CSS classes when each event happens; and by firing JavaScript events. For our purposes, we’ll be using just the CSS classes.
Implementing the WebFont Loader
As stated above, the WebFont Loader works with most web font services as well as with self-hosted fonts.
Self-hosted fonts
To use the WebFont Loader when you are hosting the font files on your own server, paste the following code into your web page:
Replace Font Family Name and Another Font Family with a comma-separated list of the font families you want to check against, and replace http://yourwebsite.com/styles.css with the URL of the style sheet where your @font-face rules reside.
Fontdeck
Assuming you have added some fonts to a website project in Fontdeck, use the afore-mentioned code for self-hosted solutions and replace http://yourwebsite.com/styles.css with the URL of the tag in your Fontdeck website settings page. It will look something like http://f.fontdeck.com/s/css/xxxx/domain/nnnn.css.
Typekit
Typekit’s JavaScript-based implementation incorporates the WebFont Loader events by default, so you won’t need to include any WebFont Loader code.
Making all browsers behave like Safari
To make Firefox and Opera work in the same way as WebKit browsers (Safari, Chrome, etc.) and Internet Explorer, and thus minimise FOUT, you need to hide the text while the fonts are loading.
While fonts are loading, the WebFont Loader adds a class of wf-loading to the element. Once the fonts have loaded, the wf-loading class is removed and replaced with a class of wf-active (or wf-inactive if all of the fonts failed to load). This means you can style elements on the page while the fonts are loading and then style them differently when the fonts have finished loading.
So, let’s say the text you need to hide while fonts are loading is contained in all paragraphs and top-level headings. By writing the following style rule into your CSS, you can hide the text while the fonts are loading:
.wf-loading h1, .wf-loading p {
visibility:hidden;
}
Because the wf-loading class is removed once the the fonts have loaded, the visibility:hidden rule will stop being applied, and the text revealed. You can see this in action on this simple example page.
That works nicely across the board, but the situation is slightly more complicated. WebKit doesn’t wait for all fonts to load before displaying text: it displays text elements as soon as the relevant font is loaded.
To emulate WebKit more accurately, we need to know when individual fonts have loaded, and apply styles accordingly. Fortunately, as mentioned earlier, the WebFont Loader has events for individual fonts too.
When a specific font is loading, a class of the form wf-fontfamilyname-n4-loading is applied. Assuming headings and paragraphs are styled in different fonts, we can make our CSS more specific as follows:
.wf-fontfamilyname-n4-loading h1,
.wf-anotherfontfamily-n4-loading p {
visibility:hidden;
}
Note that the font family name is transformed to lower case, with all spaces removed. The n4 is a shorthand for the weight and style of the font family. In most circumstances you’ll use n4 but refer to the WebFont Loader documentation for exceptions.
You can see it in action on this Safari example page (you’ll probably need to disable your cache to see any change occur).
Making all browsers behave like Firefox
To make WebKit browsers and Internet Explorer work like Firefox and Opera, you need to explicitly show text while the fonts are loading. In order to make this happen, you need to specify a font family which is not a web font while the fonts load, like this:
.wf-fontfamilyname-n4-loading h1 {
font-family: 'arial narrow', sans-serif;
}
.wf-anotherfontfamily-n4-loading p {
font-family: arial, sans-serif;
}
You can see this in action on the Firefox example page (again you’ll probably need to disable your cache to see any change occur).
And there’s more
That’s just the start of what can be done with the WebFont Loader. More areas to explore would be tweaking font sizes to reduce the impact of reflowing text and to better cater for very narrow fonts. By using the JavaScript events much more can be achieved too, such as fading in text as the fonts load.",2010,Richard Rutter,richardrutter,2010-12-02T00:00:00+00:00,https://24ways.org/2010/using-the-webfont-loader-to-make-browsers-behave-the-same/,code
307,Get the Balance Right: Responsive Display Text,"Last year in 24 ways I urged you to Get Expressive with Your Typography. I made the case for grabbing your readers’ attention by setting text at display sizes, that is to say big. You should consider very large text in the same way you might a hero image: a picture that creates an atmosphere and anchors your layout.
When setting text to be read, it is best practice to choose body and subheading sizes from a pre-defined scale appropriate to the viewport dimensions. We set those sizes using rems, locking the text sizes together so they all scale according to the page default and your reader’s preferences. You can take the same approach with display text by choosing larger sizes from the same scale.
However, display text, as defined by its purpose and relative size, is text to be seen first, and read second. In other words a picture of text. When it comes to pictures, you are likely to scale all scene-setting imagery - cover photos, hero images, and so on - relative to the viewport. Take the same approach with display text: lock the size and shape of the text to the screen or browser window.
Introducing viewport units
With CSS3 came a new set of units which are locked to the viewport. You can use these viewport units wherever you might otherwise use any other unit of length such as pixels, ems or percentage. There are four viewport units, and in each case a value of 1 is equal to 1% of either the viewport width or height as reported in reference1 pixels:
vw - viewport width,
vh - viewport height,
vmin - viewport height or width, whichever is smaller
vmax - viewport height or width, whichever is larger
In one fell swoop you can set the size of a display heading to be proportional to the screen or browser width, rather than choosing from a scale in a series of media queries. The following makes the heading font size 13% of the viewport width:
h1 {
font-size: 13 vw;
}
So for a selection of widths, the rendered font size would be:
Rendered font size (px)
Viewport width
13 vw
320
42
768
100
1024
133
1280
166
1920
250
A problem with using vw in this manner is the difference in text block proportions between portrait and landscape devices. Because the font size is based on the viewport width, the text on a landscape display is far bigger than when rendered on the same device held in a portrait orientation.
Landscape text is much bigger than portrait text when using vw units.
The proportions of the display text relative to the screen are so dissimilar that each orientation has its own different character, losing the inconsistency and considered design you would want when designing to make an impression.
However if the text was the same size in both orientations, the visual effect would be much more consistent. This where vmin comes into its own. Set the font size using vmin and the size is now set as a proportion of the smallest side of the viewport, giving you a far more consistent rendering.
h1 {
font-size: 13vmin;
}
Landscape text is consistent with portrait text when using vmin units.
Comparing vw and vmin renderings for various common screen dimensions, you can see how using vmin keeps the text size down to a usable magnitude:
Rendered font size (px)
Viewport
13 vw
13 vmin
320 × 480
42
42
414 × 736
54
54
768 × 1024
100
100
1024 × 768
133
100
1280 × 720
166
94
1366 × 768
178
100
1440 × 900
187
117
1680 × 1050
218
137
1920 × 1080
250
140
2560 × 1440
333
187
Hybrid font sizing
Using vertical media queries to set text in direct proportion to screen dimensions works well when sizing display text. In can be less desirable when sizing supporting text such as sub-headings, which you may not want to scale upwards at the same rate as the display text. For example, we can size a subheading using vmin so that it starts at 16 px on smaller screens and scales up in the same way as the main heading:
h1 {
font-size: 13vmin;
}
h2 {
font-size: 5vmin;
}
Using vmin alone for supporting text can scale it too quickly
The balance of display text to supporting text on the phone works well, but the subheading text on the tablet, even though it has been increased in line with the main heading, is starting to feel disproportionately large and a little clumsy. This problem becomes magnified on even bigger screens.
A solution to this is use a hybrid method of sizing text2. We can use the CSS calc() function to calculate a font size simultaneously based on both rems and viewport units. For example:
h2 {
font-size: calc(0.5rem + 2.5vmin);
}
For a 320 px wide screen, the font size will be 16 px, calculated as follows:
(0.5 × 16) + (320 × 0.025) = 8 + 8 = 16px
For a 768 px wide screen, the font size will be 27 px:
(0.5 × 16) + (768 × 0.025) = 8 + 19 = 27px
This results in a more balanced subheading that doesn’t take emphasis away from the main heading:
To give you an idea of the effect of using a hybrid approach, here’s a side-by-side comparison of hybrid and viewport text sizing:
table.ex--scale{width:100%;overflow: hidden;} table.ex--scale td{vertical-align:baseline;text-align:center;padding:0} tr.ex--scale-key{color:#666} tr.ex--scale-key td{font-size:.875rem;padding:0 0.125em} .ex--scale-2 tr.ex--scale-size{color:#ccc} tr.ex--scale-size td{font-size:1em;line-height:.34em;padding-bottom:.5rem} td.ex--scale-step{color:#000} td.ex--scale-hilite{color:red} .ex--scale-3 tr.ex--scale-size td{line-height:.9em}
top: calc() hybrid method; bottom: vmin only
16
20
27
32
35
40
44
16
24
38
48
54
64
72
320
480
768
960
1080
1280
1440
Over this festive period, try experiment with the proportion of rem and vmin in your hybrid calculation to see what feels best for your particular setting.
A reference pixel is based on the logical resolution of a device which takes into account double density screens such as Retina displays. ↩︎
For even more sophisticated uses of hybrid text sizing see the work of Mike Riethmuller. ↩︎",2016,Richard Rutter,richardrutter,2016-12-09T00:00:00+00:00,https://24ways.org/2016/responsive-display-text/,code
330,An Explanation of Ems,"Ems are so-called because they are thought to approximate the size of an uppercase letter M (and so are pronounced emm), although 1em is actually significantly larger than this. The typographer Robert Bringhurst describes the em thus:
The em is a sliding measure. One em is a distance equal to the type size. In 6 point type, an em is 6 points; in 12 point type an em is 12 points and in 60 point type an em is 60 points. Thus a one em space is proportionately the same in any size.
To illustrate this principle in terms of CSS, consider these styles:
#box1 {
font-size: 12px;
width: 1em;
height: 1em;
border:1px solid black;
}
#box2 {
font-size: 60px;
width: 1em;
height: 1em;
border: 1px solid black;
}
These styles will render like:
M
and
M
Note that both boxes have a height and width of 1em but because they have different font sizes, one box is bigger than the other. Box 1 has a font-size of 12px so its width and height is also 12px; similarly the text of box 2 is set to 60px and so its width and height are also 60px.",2005,Richard Rutter,richardrutter,2005-12-02T00:00:00+00:00,https://24ways.org/2005/an-explanation-of-ems/,design
286,Defending the Perimeter Against Web Widgets,"On July 14, 1789, citizens of Paris stormed the Bastille, igniting a revolution that toppled the French monarchy. On July 14 of this year, there was a less dramatic (though more tweeted) takedown: The Deck network, which delivers advertising to some of the most popular web design and culture destinations, was down for about thirty minutes. During this period, most partner sites running ads from The Deck could not be viewed as result.
A few partners were unaffected (aside from not having an ad to display). Fortunately, Dribbble, was one of them. In this article, I’ll discuss outages like this and how to defend against them. But first, a few qualifiers: The Deck has been rock solid – this is the only downtime we’ve witnessed since joining in June. More importantly, the issues in play are applicable to any web widget you might add to your site to display third-party content.
Down and out
Your defense is only as good as its weakest link. Web pages are filled with links, some of which threaten the ability of your page to load quickly and correctly. If you want your site to work when external resources fail, you need to identify the weak links on your site. In this article, we’ll talk about web widgets as a point of failure and defensive JavaScript techniques for handling them.
Widgets 101
Imagine a widget that prints out a Pun of the Day on your site. A simple technique for both widget provider and consumer is for the provider to expose a URL:
http://widgetjonesdiary.com/punoftheday.js
which returns a JavaScript file like this:
document.write(""The Pun of the Day Where do frogs go for beers after work? Hoppy hour!
"");
The call to document.write() injects the string passed into the document where it is called. So to display the widget on your page, simply add an external script tag where you want it to appear:
This approach is incredibly easy for both provider and consumer. But there are implications…
document.write()… or wrong?
As in the example above, scripts may perform a document.write() to inject HTML. Page rendering halts while a script is processed so any output can be inlined into the document. Therefore, page rendering speed depends on how fast the script returns the data. If an external JavaScript widget hangs, so does the page content that follows. It was this scenario that briefly stalled partner sites of The Deck last summer.
The elegant solution
To make our web widget more robust, calls to document.write() should be avoided. This can be achieved with a technique called JSONP (AKA JSON with padding). In our example, instead of writing inline with document.write(), a JSONP script passes content to a callback function:
publishPun(""Pun of the Day Where do frogs go for beers after work? Hoppy hour!
"");
Then, it’s up to the widget consumer to implement a callback function responsible for displaying the content. Here’s a simple example where our callback uses jQuery to write the content into a target :
…
function publishPun(content) {
$(‘.punoftheday’).html(content); // Writes content display location
}
View Example 1
Even if the widget content appears at the top of the page, our script can be included at the bottom so it’s non-blocking: a slow response leaves page rendering unaffected. It simply invokes the callback which, in turn, writes the widget content to its display destination.
The hack
But what to do if your provider doesn’t support JSONP? This was our case with The Deck. Returning to our example, I’m reminded of computer scientist David Wheeler’s statement, “All problems in computer science can be solved by another level of indirection… Except for the problem of too many layers of indirection.”
In our case, the indirection is to move the widget content into position after writing it to the page. This allows us to place the widget
Pun of the Day
Where do frogs go for beers after work? Hoppy hour!
The ‘loading-dock’ now includes the widget content, albeit hidden from view (if we’ve styled the ‘hidden’ class with display: none). There’s just one more step: move the content to its display destination. This line of jQuery (from above) does the trick:
$('.punoftheday').append($('.loading-dock').children(':gt(0)'));
This selects all child elements in the ‘loading-doc’
except the first – the widget
View Example 3
We could just add a background colour to the element using jQuery, however I prefer not to mix that information into the JavaScript as if we change the colour on our table rows I would need to remember to change it both in the CSS and in the JavaScript.
Doing something different with the last element
So here’s another thing that we often deal with. You have a list of items all floated left with a right hand margin on each element constrained within a fixed width layout. If each element has the right margin applied the margin on the final element will cause the set to become too wide forcing that last item down to the next row as shown in the below example where I have used a grey border to indicate the fixed width.
Currently we have two ways to deal with this. We can put a negative right margin on the list, the same width as the space between the elements. This means that the extra margin on the final element fills that space and the item doesn’t drop down.
The last item is different
View Example 4
The other solution will be to put a class on the final element and in the CSS remove the margin for this class.
ul.gallery li.last {
margin-right: 0;
}
This second solution may not be easy if the content is generated from server-side code that you don’t have access to change.
It could all be so different. In CSS3 we have marvellously common-sense selectors such as last-child, meaning that we can simply add rules for the last list item.
ul.gallery li:last-child {
margin-right: 0;
}
View Example 5
This removed the margin on the li which is the last-child of the ul with a class of gallery. No messing about sticking classes on the last item, or pushing the width of the item out wit a negative margin.
If this list of items repeated ad infinitum then you could also use nth-child for this task. Creating a rule that makes every 3rd element margin-less.
ul.gallery li:nth-child(3n) {
margin-right: 0;
}
View Example 6
A similar example is where the designer has added borders to the bottom of each element – but the last item does not have a border or is in some other way different. Again, only a class added to the last element will save you here if you cannot rely on using the last-child selector.
Browser support for last-child
The situation for last-child is similar to that of nth-child, in that there is no support in Internet Explorer 8. However, once again it is very simple to replicate the functionality using jQuery. Adding our .last class to the last list item.
$(""ul.gallery li:last-child"").addClass(""last"");
We could also use the nth-child selector to add the .last class to every third list item.
$(""ul.gallery li:nth-child(3n)"").addClass(""last"");
View Example 7
Fun with forms
Styling forms can be a bit of a trial, made difficult by the fact that any CSS applied to the input element will effect text fields, submit buttons, checkboxes and radio buttons. As developers we are left adding classes to our form fields to differentiate them. In most builds all of my text fields have a simple class of text whereas I wouldn’t dream of adding a class of para to every paragraph element in a document.
Syling form fields
Send your Christmas list to Santa
View Example 8
Attribute selectors provide a way of targeting elements by looking at the attributes of those elements. Unlike the other examples in this article which are CSS3 selectors, the attribute selector is actually a CSS2.1 selector – it just doesn’t get much use because of lack of support in Internet Explorer 6. Using attribute selectors we can write rules for text inputs and form buttons without needing to add any classes to the markup. For example after removing the text and button classes from my text and submit button input elements I can use the following rules to target them:
form input[type=""text""] {
border: 1px solid #333;
padding: 0.2em;
width: 400px;
}
form input[type=""submit""]{
border: 1px solid #333;
background-color: #eee;
color: #000;
padding: 0.1em;
}
View Example 9
Another problem that I encounter with forms is where I am using CSS to position my labels and form elements by floating the labels. This works fine as long as I want all of my labels to be floated, however sometimes we get a set of radio buttons or a checkbox, and I don’t want the label field to be floated. As you can see in the below example the label for the checkbox is squashed up into the space used for the other labels, yet it makes more sense for the checkbox to display after the text.
I could use a class on this label element however CSS3 lets me to target the label attribute directly by looking at the value of the for attribute.
label[for=""fOptIn""] {
float: none;
width: auto;
}
Being able to precisely target attributes in this way is incredibly useful, and once IE6 is no longer an issue this will really help to clean up our markup and save us from having to create all kinds of special cases when generating this markup on the server-side.
Browser support
The news for attribute selectors is actually pretty good with Internet Explorer 7+, Firefox 2+ and all other modern browsers all having support. As I have already mentioned this is a CSS2.1 selector and so we really should expect to be able to use it as we head into 2010! Internet Explorer 7 has slightly buggy support and will fail on the label example shown above however I discovered a workaround in the Sitepoint CSS reference comments. Adding the selector label[htmlFor=""fOptIn""] to the correct selector will create a match for IE7.
IE6 does not support these selector but, once again, you can use jQuery to plug the holes in IE6 support. The following jQuery will add the text and button classes to your fields and also add a checks class to the label for the checkbox, which you can use to remove the float and width for this element.
$('form input[type=""submit""]').addClass(""button"");
$('form input[type=""text""]').addClass(""text"");
$('label[for=""fOptIn""]').addClass(""checks"");
View Example 10
The selectors I’ve used in this article are easy to overlook as we do have ways to achieve these things currently. As developers – especially when we have frameworks and existing code that cope with these situations – it is easy to carry on as we always have done.
I think that the time has come to start to clean up our front and backend code and replace our reliance on classes with these more advanced selectors. With the help of a little JavaScript almost all users will still get the full effect and, where we are dealing with purely visual effects, there is definitely a case to be made for not worrying about the very small percentage of people with old browsers and no JavaScript. They will still receive a readable website, it may just be missing some of the finesse offered to the modern browsing experience.",2009,Rachel Andrew,rachelandrew,2009-12-20T00:00:00+00:00,https://24ways.org/2009/cleaner-code-with-css3-selectors/,code
214,Christmas Gifts for Your Future Self: Testing the Web Platform,"In the last year I became a CSS specification editor, on a mission to revitalise CSS Multi-column layout. This has involved learning about many things, one of which has been the Web Platform Tests project. In this article, I’m going to share what I’ve learned about testing the web platform. I’m also going to explain why I think you might want to get involved too.
Why test?
At one time or another it is likely that you have been frustrated by an issue where you wrote some valid CSS, and one browser did one thing with it and another something else entirely. Experiences like this make many web developers feel that browser vendors don’t work together, or they are actively doing things in a different way to one another to the detriment of those of us who use the platform. You’ll be glad to know that isn’t the case, and that the people who work on browsers want things to be consistent just as much as we do. It turns out however that interoperability, which is the official term for “works in all browsers”, is hard.
Thanks to web-platform-tests, a test from another browser vendor just found genuine bug in our code before we shipped 😻— Brian Birtles (@brianskold) February 10, 2017
In order for W3C Specifications to move on to become W3C Recommendations we need to have interoperable implementations.
6.2.4 Implementation Experience
Implementation experience is required to show that a specification is sufficiently clear, complete, and relevant to market needs, to ensure that independent interoperable implementations of each feature of the specification will be realized. While no exhaustive list of requirements is provided here, when assessing that there is adequate implementation experience the Director will consider (though not be limited to):
is each feature of the current specification implemented, and how is this demonstrated?
are there independent interoperable implementations of the current specification?
are there implementations created by people other than the authors of the specification?
are implementations publicly deployed?
is there implementation experience at all levels of the specification’s ecosystem (authoring, consuming, publishing…)?
are there reports of difficulties or problems with implementation?
https://www.w3.org/2017/Process-20170301/#transition-reqs
We all want interoperability, achieving interoperability is part of the standards process. The next question is, how do we make sure that we get it?
Unimplemented vs uninteroperable implementations
Before looking at how we can try to improve interoperability, I’d like to look at the reasons we don’t always meet that aim. There are a couple of reasons why browser X is not doing the same thing as browser Y.
The first reason is that browser X has not implemented that feature yet. Relatively small teams of people work on browser engines, and their resources are spread as thinly as those of any company. Behind those browsers are business or organisational goals which may not match our desire for a shiny feature to be made available. There are ways in which we as the web community can help gently encourage implementations - by requesting the feature, by using it so it shows up in usage reports, or writing about it to show interest. However, there will always be some degree of lag based on priorities.
A browser not supporting a feature at all, is reasonably easy to deal with these days. We can test for support with Feature Queries, and create sensible fallbacks. What is harder to deal with is when a feature is implemented in different ways by different browsers. In that situation you use the feature, perhaps referring to the specification to ensure that you are writing your CSS correctly. It looks exactly as you expect in one browser and it’s all broken when you test in another.
A frequent cause of this kind of issue is that the specification is not well defined in a particular area or that the specification has changed since one or other browser implemented it. CSS specifications are not developed in a darkened room, then presented to browser vendors to implement as a completed document. It would be nice if it worked like that, however the web platform is a gnarly thing. Before we can be sure that a specification is correct, it needs implementing in order that we can get the interoperable implementations I described earlier. A circular process has to happen. Specifications have to be written, browsers have to implement and find the problems, and then the specification has to be revised.
Many people reading this will be familiar with how flexbox changed three times in browsers, leaving us with a mess of incompatibilities and the need to use at least two versions of the spec. This story was an example of this circular process, in this case the specification was flagged as experimental using vendor prefixes. We had become used to using vendor prefixes in production and early adopters of flexbox were bitten by this. Today, specifications are developed behind experimental flags as we saw with CSS Grid Layout. Yet there has to come a time when implementations ship, and remove those flags, and it may be that knowingly or unknowingly some interop issues slip through.
You will know these interop issues as “browser bugs”, perhaps you have even reported one (thank you!) and none of us want them, so how do we make the platform as robust as possible?
How do we ensure we have interoperability?
If you were working on a large web application, with several people committing code, it would be very easy for one person to make a change that inadvertently broke some part of the application. They might not realise the fact that their change would cause a problem, due to not having a complete understanding of the entire codebase. To prevent this from happening, it is accepted good practice to write tests as well as code. The tests can then be run before the application is deployed.
Unless you start out from the beginning writing tests, and are very good at writing a test for every bit of code, it is likely that some issues do slip through from time to time. When this happens, a good approach is to not only fix the issue but also to write a test that would stop it ever happening again. That way the test suite improves over time and hopefully fewer issues happen.
The web platform is essentially a giant, sprawling application, with a huge range of people working on it in different ways. There is therefore plenty of opportunity for issues to creep in, so it seems like having some way of writing tests and automating those tests on browsers would be a good thing. That, is what the Web Platform Tests project has set out to achieve.
Web Platform Tests
Web Platform Tests is the test suite for the web platform. It is set of tests for all parts of the web platform, which can be run in any browser and the results reported. This article mostly discusses CSS tests, because I work on CSS. You will find that there are tests covering the full platform, and you can look into whichever area you have the most interest and experience in.
If we want to create a test suite for a CSS specification then we need to ensure that every feature of the specification has a related test. If a change is made to the spec, and a test committed that reflects that change, then it should be straightforward to run that test against each browser and see if it passes.
Currently, at the CSS Working Group, specifications that are at Candidate Recommendation Status should commit at test with every normative change to the spec. To explain the terminology, a normative change is one that changes some of the normative text of a specification - text that contains instructions as to how a browser should render a certain thing. A Candidate Recommendation is the status at which the Working Group officially request implementations of the spec, therefore it is reasonable to assume that any change may cause an interoperability issue. It is usually the case that representatives from all browsers will have discussed the change, so anyone who needs to change code will be aware. In this case the test allows them to check that their change passes and matches everyone else. Tests would also highlight the situation where a change to the spec caused an issue in a browser that otherwise wouldn’t be aware if it. Just as a test suite for your web application should alert a person committing code, that their change will cause a problem elsewhere.
Discovering the tests
I’ve found that the more I have understood the effort that goes into interoperability, and the reasons why creating an interoperable web is so hard, running into browser issues has become less frustrating. I have somewhere to go, even if all I can do is log the bug.
If you are even slightly interested in the subject, go have a poke around wpt.fyi. You can explore the various parts of the web platform and see how many tests have been committed. All the the CSS tests are under the directory /css where you will find each specification. Find a specification you are interested in, and look at the tests. Each test has a link to run it in your own browser to see if it passes. This can be useful in itself, if you are battling with an issue and have reduced it down to something specific, you can go and look to see if there is a test covering that and whether it appears to pass or fail in the browser you are battling with. If it turns out that the test fails, it’s probably not you!
A test on the wpt.fyi dashboard
Note: In some tests you will come across mention of a font called Ahem. This is a font designed for testing which contains consistent glyphs. You can read about how to use the font and download it here.
Contributing to Web Platform Tests
You can also become involved with Web Platform Tests. People often ask me how they can become involved in CSS, and I can think of no better way than by writing tests. You need to really understand a feature to accurately come up with a method of testing if it works or not in the different engines. This is not glamorous work, it is however a very useful thing to be involved with.
In addition to helping yourself, and developing the sort of deep knowledge of the platform that enables contribution, you will really help the progress of specifications. There are only a very few people writing specs. If those people also have to write and review all of the tests it slows down their work. If you want a better, more interoperable web, and to massively improve your ability to have nerdy conversations about highly specific things, testing is the place to start.
A local testing setup
Your first stop should be to visit the home of Web Platform Tests. There is some documentation here, which does tend to assume you know about the tests and what you are looking for - having read this article you know as much as I do. To be able to work on tests you will want to:
Clone the WPT repo, this is where all the tests are stored
Install some tools so you can run up a local copy of the tests
The instructions on the Readme in the repo should get you up and running, you can then load your own version of the test suite in a browser at http://web-platform-test:8000, or whichever port you set up.
Running tests locally
Finding things to test
It’s currently not straightforward to locate low-hanging fruit in order to start committing tests. There are some issues flagged up as a good first issue in the GitHub repo, if any of those match your interest and knowledge. Otherwise, a good place to start is where you know of existing interoperability issues. If you are aware of a browser bug, have a look and see if there is a test that addresses it. If not, then a test highlights the interoperability issue, and if it is an issue that you are running into means that you have a nice way to see if it has been fixed!
Talk to people
There is an IRC channel at irc://irc.w3.org:6667/testing, where you will find people who are writing tests as well as people who are working on the test suite framework itself. They have always been very friendly, and are likely to welcome people with a real interest in creating tests.
Gathering information
First you need to read the spec. To be able to create a test you need to know and to understand what the specification says should be happening. As I mentioned, writing tests will improve your knowledge dramatically! In general I find that web developers assume their favourite browser has got it right, this isn’t about right or wrong however, or good browsers versus bad ones. The browser with the incorrect implementation may have had a perfect, as per the spec implementation, until something changed. Do some investigation and work out what the spec says, and which – if any – browser is doing it correctly.
Another good place to look when trying to create a test for an interop issue, is to look at the browser issue trackers. It is quite likely that someone has already logged the issue, and detailed what it is, and even which browsers are as per the spec. This is useful information, as you then have a clue as to which browsers should pass your test. Remember to check version numbers - an issue may well be fixed in a pre-release version of Chrome for example, but not in the public release.
Edge Issue Tracker
Mozilla Issue Tracker
WebKit Issue Tracker
Chromium Issue Tracker
Writing the test
If you’ve ever created a Reduced Test Case to isolate a browser issue, you already have some idea of what we are trying to do with a test. We want to test one thing, in isolation, and to be able to confirm “yes this works as per the spec” or “no, this does not”.
The main two types of test are:
testharness.js tests
reftests
The testharness.js tests use JavaScript to test an assertion, this framework is designed as a way to test Web APIs and as this quickly gets fairly complicated - and I’m a complete beginner myself at writing these - I’ll refer you to the excellent tutorial Using testharness.js.
Many CSS tests will be reftests. A reftest involves getting two pages to lay out in the same way, so that they are visually the same. For example, you can find a reftest for Grid Layout at:https://w3c-test.org/css/css-grid/alignment/grid-gutters-001.html or at http://web-platform.test:8000/css/css-grid/alignment/grid-gutters-001.html if you have run up your own copy of WPT.
CSS Grid Layout Test: Support for gap shorthand property of row-gap and column-gap
The test passes if it has the same visual effect as reference.
I am testing the new gap property (renamed grid-gap). The reference file can be found by looking for the line:
In that file, I am using absolute positioning to mock up the way the file would look if gap is implemented correctly.
CSS Grid Layout Reference: a square with a green cross
The tests are compared in an automated way by taking screenshots of the test and reference.
These are relatively simple tests to write, you will find the work is not in writing the test however. The work is really in doing the research, and making sure you understand what is supposed to happen so you can write the test. Which is why, if you really want to get your hands dirty in the web platform, this is a good place to start.
Committing a test
Once you have written a test you can run the lint tool to make sure that everything is tidy. This tool is run automatically after you submit your pull request, and reviewers won’t accept a test with lint errors, so do this locally first to catch anything obvious.
Tests are added as a pull request, once you have your test ready to go you can create a pull request to add it to the repository. Your test will be tested and it will then wait for a review.
You may well then find yourself in a bit of a waiting game, as the test needs to be reviewed. How long that takes will depend on how active work is on that spec. People who are in the OWNERS file for that spec should be notified. You can always ask in IRC to see if someone is available who can look at and potentially merge your test.
Usually the reviewer will have some comments as to how the test can be improved, in the same as the owner of an open source project you submit a PR to might ask you to change some things. Work with them to make your test as good as it can be, the things you learn on the first test you submit will make future ones easier. You can then bask in the glow of knowing you have done something towards the aim of a more interoperable web for all of us.
Christmas gifts for your future self
I have been a web developer for over 20 years. I have no idea what the web platform will look like in 20 more years, but for as long as I’m working on it I’ll keep on trying to make it better. Making the web more interoperable makes it a better place to be a web developer, storing up some Christmas gifts for my future self, while learning new things as I do so.
Resources
I rounded up everything I could find on WPT while researching this article. As well as some other links that might be helpful for you. These links are below. Happy testing!
Web Platform Tests
Using testharness.js
IRC Channel irc://irc.w3.org:6667/testing
Edge Issue Tracker
Mozilla Issue Tracker
WebKit Issue Tracker
Chromium Issue Tracker
Reducing an Issue - guide to created a reduced test case
Effectively Using Web Platform Tests: Slides and Video
An excellent walkthrough from Lyza Gardner on her working on tests for the HTML specification - Moving Targets: a case study on testing web standards.
Improving interop with web-platform-tests: Slides and Video",2017,Rachel Andrew,rachelandrew,2017-12-10T00:00:00+00:00,https://24ways.org/2017/testing-the-web-platform/,code
243,Researching a Property in the CSS Specifications,"I frequently joke that I’m “reading the specs so you don’t have to”, as I unpack some detail of a CSS spec in a post on my blog, some documentation for MDN, or an article on Smashing Magazine. However waiting for someone like me to write an article about something is a pretty slow way to get the information you need. Sometimes people like me get things wrong, or specifications change after we write a tutorial.
What if you could just look it up yourself? That’s what you get when you learn to read the CSS specifications, and in this article my aim is to give you the basic details you need to grab quick information about any CSS property detailed in the CSS specs.
Where are the CSS Specifications?
The easiest way to see all of the CSS specs is to take a look at the Current Work page in the CSS section of the W3C Website. Here you can see all of the specifications listed, the level they are at and their status. There is also a link to the specification from this page. I explained CSS Levels in my article Why there is no CSS 4.
Who are the specifications for?
CSS specifications are for everyone who uses CSS. You might be a browser engineer - referred to as an implementor - needing to know how to implement a feature, or a web developer - referred to as an author - wanting to know how to use the feature. The fact that both parties are looking at the same document hopefully means that what the browser displays is what the web developer expected.
Which version of a spec should I look at?
There are a couple of places you might want to look. Each published spec will have the latest published version, which will have TR in the URL and can be accessed without a date (which is always the newest version) or at a date, which will be the date of that publication. If I’m referring to a particular Working Draft in an article I’ll typically link to the dated version. That way if the information changes it is possible for someone to see where I got the information from at the time of writing.
If you want the very latest additions and changes to the spec, then the Editor’s Draft is the place to look. This is the version of the spec that the editors are committing changes to. If I make a change to the Multicol spec and push it to GitHub, within a few minutes that will be live in the Editor’s Draft. So it is possible there are errors, bits of text that we are still working out and so on. The Editor’s Draft however is definitely the place to look if you are wanting to raise an issue on a spec, as it may be that the issue you are about to raise is already fixed.
If you are especially keen on seeing updates to specifications keep an eye on https://drafts.csswg.org/ as this is a list of drafts, along with the date they were last updated.
How to approach a spec
The first thing to understand is that most CSS Specifications start with the most straightforward information, and get progressively further into the weeds. For an author the initial examples and explanations are likely to be of interest, and then the property definitions and examples. Therefore, if you are looking at a vast spec, know that you probably won’t need to read all the way to the bottom, or read every section in detail.
The second thing that is useful to know about modern CSS specifications is how modularized they are. It really never is a case of finding everything you need in a single document. If we tried to do that, there would be a lot of repetition and likely inconsistency between specs. There are some key specifications that many other specifications draw on, such as:
Values and Units
Intrinsic and Extrinsic Sizing
Box Alignment
When something is defined in another specification the spec you are reading will link to it, so it is worth opening that other spec in a new tab in order that you can refer back to it as you explore.
Researching your property
As an example we will take a look at the property grid-auto-rows, this property defines row tracks in the implicit grid when using CSS Grid Layout. The first thing you will need to do is find out which specification defines this property.
You might already know which spec the property is part of, and therefore you could go directly to the spec and search using your browser or look in the navigation for the spec to find it. Alternatively, you could take a look at the CSS Property Index, which is an automatically generated list of CSS Properties.
Clicking on a property will take you to the TR version of the spec, the latest published draft, and the definition of that property in it. This definition begins with a panel detailing the syntax of this property. For grid-auto-rows, you can see that it is listed along with grid-auto-columns as these two properties are essentially identical. They take the same values and work in the same way, one for rows and the other for columns.
Value
For value we can see that the property accepts a value
. The next thing to do is to find out what that actually means, clicking will take you to where it is defined in the Grid spec.
The value is defined as accepting various values:
minmax( , )
fit-content(
We need to head down the rabbit hole to find out what each of these mean. From here we essentially go down line by line until we have unpacked the value of track-size.
is defined just below as:
min-content
max-content
auto
So these are all things that would be valid to use as a value for grid-auto-rows.
The first value of is something you will see in many specifications as a value. It means that you can use a length unit - for example px or em - or a percentage. Some properties only accept a in which case you know that you cannot use a percentage as the value. This means that you could have grid-auto-rows with any of the following values.
grid-auto-rows: 100px;
grid-auto-rows: 1em;
grid-auto-rows: 30%;
When using percentages, it is important to know what it is a percentage of. As a percentage has to resolve from something. There is text in the spec which explains how column and row percentages work.
“ values are relative to the inline size of the grid container in column grid tracks, and the block size of the grid container in row grid tracks.”
This means that in a horizontal writing mode such as when using English, a percentage when used as a track-size in grid-auto-columns would be a percentage of the width of the grid, and a percentage in grid-auto-rows a percentage of the height of the grid.
The second value of is also defined here, as “A non-negative dimension with the unit fr specifying the track’s flex factor.” This is the fr unit, and the spec links to a fuller definition of fr as this unit is only used in Grid Layout so it is therefore defined in the grid spec. We now know that a valid value would be:
grid-auto-rows: 1fr;
There is some useful information about the fr unit in this part of the spec. It is noted that the fr unit has an automatic minimum. This means that 1fr is really minmax(auto, 1fr). This is why having a number of tracks all at 1fr does not mean that all are equal sized, as a larger item in any of the tracks would have a large auto size and therefore would be larger after spare space had been distributed.
We then have min-content and max-content. These keywords can be used for track sizing and the specification defines what they mean in the context of sizing a track, representing the min and max-sizing contributions of the grid tracks. You will see that there are various terms linked in the definition, so if you do not know what these mean you can follow them to find out.
For example the spec links max-content contribution to the CSS Intrinsic and Extrinsic Sizing specification. This is one of those specs which is drawn on by many other specifications. If we follow that link we can read the definition there and follow further links to understand what each term means. The more that you read specifications the more these terms will become familiar to you. Just like learning a foreign language, at first you feel like you have to look up every little thing. After a while you remember the vocabulary.
We can now add min-content and max-content to our available values.
grid-auto-rows: min-content;
grid-auto-rows: max-content;
The final item in our list is auto. If you are familiar with using Grid Layout, then you are probably aware that an auto sized track for will grow to fit the content used. There is an interesting note here in the spec detailing that auto sized rows will stretch to fill the grid container if there is extra space and align-content or justify-content have a value of stretch. As stretch is the default value, that means these tracks stretch by default. Tracks using other types of length will not behave like this.
grid-auto-rows: auto;
So, this was the list for , the next possible value is minmax( , ). So this is telling us that we can use minmax() as a value, the final (max) value will be and we have already unpacked all of the allowable values there. The first value (min) is detailed as an . If we look at the values for this, we discover that they are the same as , minus the value:
min-content
max-content
auto
We already know what all of these do, so we can add possible minmax() values to our list of values for .
grid-auto-rows: minmax(100px, 200px);
grid-auto-rows: minmax(20%, 1fr);
grid-auto-rows: minmax(1em, auto);
grid-auto-rows: minmax(min-content, max-content);
Finally we can use fit-content( . We can see that fit-content takes a value of which we already know to be either a length unit, or a percentage. The spec details how fit-content is worked out, and it essentially allows a track which acts as if you had used the max-content keyword, however the track stops growing when it hits the length passed to it.
grid-auto-rows: fit-content(200px);
grid-auto-rows: fit-content(20%);
Those are all of our possible values, and to round things off, check again at the initial value, you can see it has a little + sign next to it, click that and you will be taken to the CSS Values and Units module to find that, “A plus (+) indicates that the preceding type, word, or group occurs one or more times.” This means that we can pass a single track size to grid-auto-rows or multiple track sizes as a space separated list. Below the box is an explanation of what happens if you pass in more than one track size:
“If multiple track sizes are given, the pattern is repeated as necessary to find the size of the implicit tracks. The first implicit grid track after the explicit grid receives the first specified size, and so on forwards; and the last implicit grid track before the explicit grid receives the last specified size, and so on backwards.”
Therefore with the following CSS, if five implicit rows were needed they would be as follows:
100px
1fr
auto
100px
1fr
.grid {
display: grid;
grid-auto-rows: 100px 1fr auto;
}
Initial
We can now move to the next line in the box, and you’ll be glad to know that it isn’t going to require as much unpacking! This simply defines the initial value for grid-auto-rows. If you do not specify anything, created rows will be auto sized. All CSS properties have an initial value that they will use if they are invoked as part of the usage of the specification they are in, but you do not set a value for them. In the case of grid-auto-rows it is used whenever rows are created in the implicit grid, so it needs to have a value to be used even if you do not set one.
Applies to
This line tells us what this property is used for. Some properties are used in multiple places. For example if you look at the definition for justify-content in the Box Alignment specification you can see it is used in multicol containers, flex containers, and grid containers. In our case the property only applies for grid containers.
Inherited
This tells us if the property can be inherited from a parent element if it is not set. In the case of grid-auto-rows it is not inherited. A property such as color is inherited, so you do not need to set it on each element.
Percentages
Are percentages allowed for this property, and if so how are they calculated. In this case we are referred to the definition for grid-template-columns and grid-template-rows where we discover that the percentage is from the corresponding dimension of the content area.
Media
This defines the media group that the property belongs to. In this case visual.
Computed Value
This details how the value is resolved. The grid-auto-rows property again refers to track sizing as defined for grid-template-columns and grid-template-rows, which tells us the computed value is as specified with lengths made absolute.
Canonical Order
If you have a property–generally a shorthand property–which takes multiple values in a set order, then those values need to be serialized in the order detailed in the grammar for that property. In general you don’t need to worry about this value in the table.
Animation Type
This details whether the property can be animated, and if so what type of animation. This is useful if you are trying to animate something and not getting the result that you expect. Note that just because something is listed in the spec as animatable does not mean that browsers will have implemented animation for that property yet!
That’s (mostly) it!
Sometimes the property will have additional examples - there is one underneath the table for grid-auto-rows. These are worth looking at as they will highlight usage of the property that the spec editor has felt could use an example. There may also be some additional text explaining anythign specific to this property.
In selecting grid-auto-rows I chose a fairly complex property in terms of the work we needed to do to unpack the value. Many properties are far simpler than this. However ultimately, even when you come across a complex value, it really is just a case of stepping through the definitions until you come to the bottom of the rabbit hole.
Being able to work out what is valid for each property is incredibly useful. It means you don’t waste time trying to use a value that doesn’t work for that property. You also may find that there are values you weren’t aware of, that solve problems for you.
Further reading
Specifications are not designed to be user manuals, and while they often contain examples, these are pretty terse as they need to be clear to demonstrate their particular point. The manual for the Web Platform is MDN Web Docs. Pairing reading a specification with the examples and information on an MDN property page such as the one for grid-auto-rows is a really great way to ensure that you have all the information and practical usage examples you might need.
You may also find useful:
Value Definition Syntax on MDN.
The MDN Glossary defines many common terms.
Understanding the CSS Property Value Syntax goes into more detail in terms of reading the syntax.
How to read W3C Specs - from 2001 but still relevant.
I hope this article has gone some way to demystify CSS specifications for you. Even if the specifications are not your preferred first stop to learn about new CSS, being able to go directly to the source and avoid having your understanding filtered by someone else, can be very useful indeed.",2018,Rachel Andrew,rachelandrew,2018-12-14T00:00:00+00:00,https://24ways.org/2018/researching-a-property-in-the-css-specifications/,code
306,What next for CSS Grid Layout?,"In 2012 I wrote an article for 24 ways detailing a new CSS Specification that had caught my eye, at the time with an implementation only in Internet Explorer. What I didn’t realise at the time was that CSS Grid Layout was to become a theme on which I would base the next four years of research, experimentation, writing and speaking.
As I write this article in December 2016, we are looking forward to CSS Grid Layout being shipped in Chrome and Firefox. What will ship early next year in those browsers is expanded and improved from the early implementation I explored in 2012. Over the last four years the spec has been developed as part of the CSS Working Group process, and has had input from browser engineers, specification writers and web developers. Use cases have been discussed, and features added.
The CSS Grid Layout specification is now a Candidate Recommendation. This status means the spec is to all intents and purposes, finished. The discussions now happening are on fine implementation details, and not new feature ideas. It makes sense to draw a line under a specification in order that browser vendors can ship complete, interoperable implementations. That approach is good for all of us, it makes development far easier if we know that a browser supports all of the features of a specification, rather than working out which bits are supported. However it doesn’t mean that works stops here, and that new use cases and features can’t be proposed for future levels of Grid Layout. Therefore, in this article I’m going to take a look at some of the things I think grid layout could do in the future. I would love for these thoughts to prompt you to think about how Grid - or any CSS specification - could better suit the use cases you have.
Subgrid - the missing feature of Level 1
The implementation of CSS Grid Layout in Chrome, Firefox and Webkit is comparable and very feature complete. There is however one standout feature that has not been implemented in any browser as yet - subgrid. Once you set the value of the display property to grid, any direct children of that element become grid items. This is similar to the way that flexbox behaves, set display: flex and all direct children become flex items. The behaviour does not apply to children of those items. You can nest grids, just as you can nest flex containers, but the child grids have no relationship to the parent.
Nesting Grids by Rachel Andrew (@rachelandrew) on CodePen.
The subgrid behaviour would enable the grid defined on the parent to be used by the children. I feel this would be most useful when working with a multiple column flexible grid - for example a typical 12 column grid. I could define a grid on a wrapper, then position UI elements on that grid - from the major structural elements of my page down through the child elements to a form where I wanted the field to line up with items above.
The specification contained an initial description of subgrid, with a value of subgrid for grid-template-columns and grid-template-rows, you can read about this in the August 2015 Working Draft. This version of the specification would have meant you could declare a subgrid in one dimension only, and create a different set of tracks in the other.
In an attempt to get some implementation of subgrid, a revised specification was proposed earlier this year. This gives a single subgrid value of the display property. As we now cannot specify a subgrid on rows OR columns this limits us to have a subgrid that works in two dimensions. At this point neither version has been implemented by anyone, and subgrids are marked as “at risk” in the Level 1 Candidate Recommendation. With regard to ‘at-risk’ this is explained as follows:
“‘At-risk’ is a W3C Process term-of-art, and does not necessarily imply that the feature is in danger of being dropped or delayed. It means that the WG believes the feature may have difficulty being interoperably implemented in a timely manner, and marking it as such allows the WG to drop the feature if necessary when transitioning to the Proposed Rec stage, without having to publish a new Candidate Rec without the feature first.”
If we lose subgrid from Level 1, as it looks likely that we will, this does give us a chance to further discuss and iterate on that feature. My current thoughts are that I’m not completely happy about subgrids being tied to both dimensions and feel that a return to the earlier version, or something like it, would be preferable.
Further reading about subgrid
My post from 2015 detailing why I feel subgrid is important
My post based on the revised specification
Eric Meyer’s thoughts on subgrid
Write-up of a discussion from Igalia who work on the Blink and Webkit browser implementations
Styling cells, tracks and areas
Having defined a grid with CSS Grid Layout you can place child elements into that grid, however what you can’t do is style the grid tracks or cells. Grid doesn’t even go as far as multiple column layout, which has the column-rule properties.
In order to set a background colour on a grid cell at the moment you would have to add an empty HTML element or insert some generated content as in the below example. I’m using a 1 pixel grid gap to fake lines between grid cells, and empty div elements, and some generated content to colour those cells.
Faked backgrounds and borders by Rachel Andrew (@rachelandrew) on CodePen.
I think it would be a nice addition to Grid Layout to be able to directly add backgrounds and borders to cells, tracks and areas. There is an Issue raised in the CSS WG Drafts repository for Decorative Grid Cell pseudo-elements, if you want to add thoughts to that.
More control over auto placement
If you haven’t explicitly placed the direct children of your grid element they will be laid out according to the grid auto placement rules. You can see in this example how we have created a grid and the items are placing themselves into cells on that grid.
Items auto-place on a defined grid by Rachel Andrew (@rachelandrew) on CodePen.
The auto-placement algorithm is very cool. We can position some items, leaving others to auto-place; we can set items to span more than one track; we can use the grid-auto-flow property with a value of dense to backfill gaps in our grid.
Websafe colors meet CSS Grid (auto-placement demo) by Rachel Andrew (@rachelandrew) on CodePen.
I think however this could be taken further. In this issue posted to my CSS Grid AMA on GitHub, the question is raised as to whether it would be possible to ask grid to place items on the next available line of a certain name. This would allow you to skip tracks in the grid when using auto-placement, an issue that has also been raised by Emil Björklund in this post to the www-style list prior to spec discussion moving to Github. I think there are probably similar issues, if you can think of one add a comment here.
Creating non-rectangular grid areas
A grid area is a collection of grid cells, defined by setting the start and end lines for columns and rows or by creating the area in the value of the grid-template-areas property as shown below. Those areas however must be rectangular - you can’t create an L-shaped or otherwise non-regular shape.
Grid Areas by Rachel Andrew (@rachelandrew) on CodePen.
Perhaps in the future we could define an L-shape or other non-rectangular area into which content could flow, as in the below currently invalid code where a quote is embedded into an L-shaped content area.
.wrapper {
display: grid;
grid-template-areas:
""sidebar header header""
""sidebar content quote""
""sidebar content content"";
}
Flowing content through grid cells or areas
Some uses cases I have seen perhaps are not best solved by grid layout at all, but would involve grid working alongside other CSS specifications. As I detail in this post, there are a class of problems that I believe could be solved with the CSS Regions specification, or a revised version of that spec.
Being able to create a grid layout, then flow content through the areas could be very useful. Jen Simmons presented to the CSS Working Group at the Lisbon meeting a suggestion as to how this might work.
In a post from earlier this year I looked at a collection of ideas from specifications that include Grid, Regions and Exclusions. These working notes from my own explorations might prompt ideas of your own.
Solving the keyboard/layout disconnect
One issue that grid, and flexbox to a lesser extent, raises is that it is very easy to end up with a layout that is disconnected from the underlying markup. This raises problems for people navigating using the keyboard as when tabbing around the document you find yourself jumping to unexpected places. The problem is explained by Léonie Watson with reference to flexbox in Flexbox and the keyboard navigation disconnect.
The grid layout specification currently warns against creating such a disconnect, however I think it will take careful work by web developers in order to prevent this. It’s also not always as straightforward as it seems. In some cases you want the logical order to follow the source, and others it would make more sense to follow the visual. People are thinking about this issue, as you can read in this mailing list discussion.
Bringing your ideas to the future of Grid Layout
When I’m not getting excited about new CSS features, my day job involves working on a software product - the CMS that is serving this very website, Perch. When we launched Perch there were many use cases that we had never thought of, despite having a good idea of what might be needed in a CMS and thinking through lots of use cases. The additional use cases brought to our attention by our customers and potential customers informed the development of the product from launch. The same will be true for Grid Layout.
As a “product” grid has been well thought through by many people. Yet however hard we try there will be use cases we just didn’t think of. You may well have one in mind right now. That’s ok, because as with any CSS specification, once Level One of grid is complete, work can begin on Level Two. The feature set of Level Two will be informed by the use cases that emerge as people get to grips with what we have now.
This is where you get to contribute to the future of layout on the web. When you hit up against the things you cannot do, don’t just mutter about how the CSS Working Group don’t listen to regular developers and code around the problem. Instead, take a few minutes and write up your use case. Post it to your blog, to Medium, create a CodePen and go to the CSS Working Group GitHub specs repository and post an issue there. Write some pseudo-code, draw a picture, just make sure that the use case is described in enough detail that someone can see what problem you want grid to solve. It may be that - as with any software development - your use case can’t be solved in exactly the way you suggest. However once we have a use case, collected with other use cases, methods of addressing that class of problems can be investigated.
I opened this article by explaining I’d written about grid layout four years ago, and how we’re only now at a point where we will have Grid Layout available in the majority of browsers. Specification development, and implementation into browsers takes time. This is actually a good thing, as it’s impossible to take back CSS once it is out there and being used by production websites. We want CSS in the wild to be well thought through and that takes time. So don’t feel that because you don’t see your use case added to a spec immediately it has been ignored. Do your future self a favour and write down your frustrations or thoughts, and we can all make sure that the web platform serves the use cases we’re dealing with now and in the future.",2016,Rachel Andrew,rachelandrew,2016-12-12T00:00:00+00:00,https://24ways.org/2016/what-next-for-css-grid-layout/,code
332,CSS Layout Starting Points,"I build a lot of CSS layouts, some incredibly simple, others that cause sleepless nights and remind me of the torturous puzzle books that were given to me at Christmas by aunties concerned for my education. However, most of the time these layouts fit quite comfortably into one of a very few standard formats. For example:
Liquid, multiple column with no footer
Liquid, multiple column with footer
Fixed width, centred
Rather than starting out with blank CSS and (X)HTML documents every time you need to build a layout, you can fairly quickly create a bunch of layout starting points, that will give you a solid basis for creating the rest of the design and mean that you don’t have to remember how a three column layout with a footer is best achieved every time you come across one!
These starting points can be really basic, in fact that’s exactly what you want as the final design, the fonts, the colours and so on will be different every time. It’s just the main sections we want to be able to quickly get into place. For example, here is a basic starting point CSS and XHTML document for a fixed width, centred layout with a footer.
Fixed Width and Centred starting point document
Your main content goes here.
body {
text-align: center;
min-width: 740px;
padding: 0;
margin: 0;
}
#wrapper {
text-align: left;
width: 740px;
margin-left: auto;
margin-right: auto;
padding: 0;
}
#content {
margin: 0 200px 0 0;
}
#content .inner {
padding-top: 1px;
margin: 0 10px 10px 10px;
}
#side {
float: right;
width: 180px;
margin: 0;
}
#side .inner {
padding-top: 1px;
margin: 0 10px 10px 10px;
}
#footer {
margin-top: 10px;
clear: both;
}
#footer .inner {
margin: 10px;
}
9 times out of 10, after figuring out exactly what main elements I have in a layout, I can quickly grab the ‘one I prepared earlier’, mark-up the relevant sections within the ready-made divs, and from that point on, I only need to worry about the contents of those different areas. The actual layout is tried and tested, one that I know works well in different browsers and that is unlikely to throw me any nasty surprises later on. In addition, considering how the layout is going to work first prevents the problem of developing a layout, then realising you need to put a footer on it, and needing to redevelop the layout as the method you have chosen won’t work well with a footer.
While enjoying your mince pies and mulled wine during the ‘quiet time’ between Christmas and New Year, why not create some starting point layouts of your own? The css-discuss Wiki, CSS layouts section is a great place to find examples that you can try out and find your favourite method of creating the various layout types.",2005,Rachel Andrew,rachelandrew,2005-12-04T00:00:00+00:00,https://24ways.org/2005/css-layout-starting-points/,code
199,Knowing the Future - Tips for a Happy Launch Day,"You’ve chosen your frameworks and libraries. You’ve learned how to write code which satisfies the buzzword and performance gods. Now you need to serve it to a global audience, and make things easy to preview, to test, to sign-off, and to evolve.
But infrastructure design is difficult and boring for most of us. We just want to get our work out into the wild.
If only we had tools which would let us go, “Oh yeah! It all deploys perfectly every time” and shout, “You need another release? BAM! What’s next?”
A truth that can be hard to admit is that very often, the production environment and its associated deployment processes are poorly defined until late into a project. This can be a problem.
It makes my palms sweaty just thinking about it.
If like me, you have spent time building things for clients, you’ll probably have found yourself working with a variety of technical partners and customers who bring different constraints and opportunities to your projects. Knowing and proving the environments and the deployment processes is often very difficult, but can be a factor which profoundly impacts our ability to deliver what we promised. To say nothing of our ability to sleep at night or leave our fingernails un-chewed.
Let’s look at this a little, and see if we can’t set you up for a good night’s sleep, with dry palms and tidy fingernails.
A familiar problem
You’ve been here too, right? The project development was tough, but you’re pleased with what you are running in your local development environments. Now you need to get the client to see and approve your build, and hopefully indicate with a cheery thumbs up that it can “go live”.
Chances are that we have a staging environment where the client can see the build. But be honest, is this exactly the same as the production environment? It should be, but often it’s not. Often the staging environment is nothing more than a visible server with none of the optimisations, security, load balancing, caching, and other vital bits of machinery that we’ll need (and need to test) in “prod”.
Often the production environment is still being “set up” and you’ll have to wait and see.
In development, “wait and see” is the enemy.
Instead of waiting to see, we need to make the provisioning of, and deployment to our different environments one of the very first jobs of our project. I’ve often needed to be the unpopular voice in the room who makes a big fuss when this is delayed. I’ve described it as being a “critical blocker” during project meetings and suggested that everything should halt until it is fixed.
It is that important.
Clients don’t often like hearing a wary, disruptive voice saying “whoa there Nelly!”, because the development should be able to continue while the production environment gets sorted out, right?
Sure. But if it is not seen as a blocker, it is seen as something that can just happen later. And if it happens later, all the ugly surprises and unknowns surface later too. And later is when we’ll need to be thinking about other things. Not the plumbing. Trust me, it pays to face up to the issue right away rather than press on optimistically. The client will thank you later.
Attitudes and expectations
We should, I think, exhibit these four attitudes towards production deployment:
Make it scripted
Make it automated
Make it real
Make it first
Make it scripted
Let’s face it, we are going to need to deploy more than once over the course of the project. We are not going to get things perfect on our first shot. Nor should we expect to. And if we are going to repeat something, we want to be able to do it identically and predictably every time without needing to rely on our memories.
Developers are great at scripting things which they would otherwise need to repeat. It makes us faster and it also helps us keep track of the steps we need to take.
I’m not crazy enough to try suggest the best technology to script your builds or deployments (holy wars lie down that path). A lot will depend on your languages and your tastes. Some will like Fabric, others will prefer Gulp, you might prefer Make or NPM. It doesn’t really matter as long as you can script the process of building, packaging and deploying your project.
Wait. Won’t we need to know everything about the build from the start in order to do this? Aren’t our dependencies likely to change over time?
Yes. That would be ideal. But it’s ok. Like our code, our deployment script will evolve over the life of a project. So evolve it. Start by scripting what is needed to support the first iteration of the project, and then maintain that script. It will become a valuable “source of truth”, providing a form of documentation of what your project needs for a successful deployment. Another bonus.
Make it automated
If we have a scripted deployment which we can run by executing a single command, then we are in great shape to automate that process by triggering the build and deployment via suitable events.
Again, I prefer not to offer one single suggestion of when this should occur. That will depend on your approach to the project, how your development team is organised, and how your QA team operate. You can tune this to suit.
For one project I worked on, we chose to trigger the build and deployment to our production environment every time we used Git to tag the master branch of our version control repository. There were a few moving parts, and we needed to do some upfront work to get everything working, but that upfront effort was repaid many fold as we deployed time and time again, and exposed some issues with our environment long before we got to “launch day”.
With a scripted and automated process, we can make deployments “cheap”. This is our goal. When there are minimal cognitive or time overheads associated with deploying, we’re likely to do it all the more often and become more confident that it will behave as expected.
Make it real
Alright, we have written scripts to build and deploy our projects. Anyone tagging our repo will trigger things to happen as if by magic, but where are we pushing things to? We need to target a real environment if this is to have any value.
A useful pattern is to have all activity on our develop branch trigger deployments to our staging server. Meanwhile tagging master will deploy a version to the production environment. How we organise this will depend on our git branching approach. (I’ve seen as many ways of approaching Git Flow as I have seen ways of approaching “Agile”).
It’s vital though, that we ensure that we are deploying to, and testing against, our real infrastructure. We want to see real results. That’s the best way to learn real lessons.
Make it first
Building our site to run in an environment not yet fully defined or available to test is like climbing without ropes – it’s possible, but we put ourselves at risk. And the higher we climb the greater the risk. So it is important to do this as early as we possibly can.
Don’t have a certificate for our HTTPS yet? Fine, but let’s still deploy to this evolving production environment and introduce HTTPS as soon as we can.
Before we know it we’ll be proving that this is set up correctly and we’ll not be surprised by mixed security alerts or other nasties further down the line.
Mailchimp perfectly capture the anxiety of sending emails to gazillions of people for a campaign. But we’re lucky. Launching a site doesn’t need to be like performing a mailshot. We can do things to banish that sweaty hand.
Doing preparation work upfront means that by the time we need to launch the site into the wild, we have exercised the deployment mechanics, and tested the production environment so rigorously that this task will be boring.
(It won’t be boring. Launching should always be exciting because the world will finally get to see our beautiful, painstaking work. But nor should it be terrifying. Especially as a result of not knowing for certain if our processes and environments are going to work or burst into flames on the big day.)
What tools exist?
Well this all sounds lovely. But how should we tackle this? Where are the tools for us to use? As it happens, there are many service and tools that we can use to work this way.
Hosting
All of the big players like Amazon, Azure and Google offer tools which can help us here. Google for example, can host multiple deployed versions of your project in parallel and you can manage them via their App Engine console. Each build receives its own URL which you can use to access any deployed version of your site.
Having immutable deployments which stick around in perpetuity (or until you bin them) is a key feature which unlocks the ability to confidently direct your traffic to any version of your site. With that comes the capacity to test any version or feature in its real environment, and then promote a version, or rollback to a previous version whenever you want.
A liberating power to have.
Continuous integration
In order to create all of those different versions, we’ll need somewhere to run our build and deployment scripts. Jenkins has been a popular Continuous Integration (CI) option for some time, and can be configured to perform all sorts of tasks, giving you extensive control over your deployment pipeline.
You need to host Jenkins yourself, but it provides some simple ways to do that.
The landscape for CI is getting richer and richer. With many hosted services like Circle CI providing this kind of automation up in the cloud.
One stop shop
Netlify combines both hosting and continuous integration services. It monitors your git repositories and automatically runs your build in a container on its servers when it finds changes. Each branch and pull request in your git repository will result in an immutable version of your site with its own URL.
Netlify is unlike Google Cloud, AWS or Azure in that it cannot host a dynamic server-side application for you. Instead it specialises in hosting static, or so called JAMstack sites.
Personally, I find that its simplicity makes it an approachable option, and a good place to learn and adopt some of these valuable habits.
Full disclosure: I’m a Netlify employee. But before I was, I was an avid customer, and it was through using Netlify that I first encountered some of these principles in practice.
Conclusion. It’s all about the approach
No matter what tools or services you use (and there are many which can support these practices), the most important thing is to adopt an approach which lets you prove your environments as quickly as possible.
Front-loading this effort will cast light onto the issues that you’ll need to address early and often, leaving no infrastructure surprises to spoil things for you on launch day.
Automating the process will mean that when you do find things that you need to fix or to improve later (and you will), issuing another release will be trivial. It is a lovely feeling when you have confidence that releasing v1.0.0 will be no more stressful v0.0.1. In fact it should actually be less stressful, as you’ll have been down this road many times by then. Fixing the potholes and smoothing the way as you went.
From here, it should be a smooth ride.",2017,Phil Hawksworth,philhawksworth,2017-12-21T00:00:00+00:00,https://24ways.org/2017/knowing-the-future/,process
121,Hide And Seek in The Head,"If you want your JavaScript-enhanced pages to remain accessible and understandable to scripted and noscript users alike, you have to think before you code. Which functionalities are required (ie. should work without JavaScript)? Which ones are merely nice-to-have (ie. can be scripted)? You should only start creating the site when you’ve taken these decisions.
Special HTML elements
Once you have a clear idea of what will work with and without JavaScript, you’ll likely find that you need a few HTML elements for the noscript version only.
Take this example: A form has a nifty bit of Ajax that automatically and silently sends a request once the user enters something in a form field. However, in order to preserve accessibility, the user should also be able to submit the form normally. So the form should have a submit button in noscript browsers, but not when the browser supports sufficient JavaScript.
Since the button is meant for noscript browsers, it must be hard-coded in the HTML:
When JavaScript is supported, it should be removed:
var checkJS = [check JavaScript support];
window.onload = function () {
if (!checkJS) return;
document.getElementById('noScriptButton').style.display = 'none';
}
Problem: the load event
Although this will likely work fine in your testing environment, it’s not completely correct. What if a user with a modern, JavaScript-capable browser visits your page, but has to wait for a huge graphic to load? The load event fires only after all assets, including images, have been loaded. So this user will first see a submit button, but then all of a sudden it’s removed. That’s potentially confusing.
Fortunately there’s a simple solution: play a bit of hide and seek in the :
var checkJS = [check JavaScript support];
if (checkJS) {
document.write('');
}
First, check if the browser supports enough JavaScript. If it does, document.write an extra