rowid,title,contents,year,author,author_slug,published,url,topic 256,Develop Your Naturalist Superpowers with Observable Notebooks and iNaturalist,"We’re going to level up your knowledge of what animals you might see in an area at a particular time of year - a skill every naturalist* strives for - using technology! Using iNaturalist and Observable Notebooks we’re going to prototype seasonality graphs for particular species in an area, and automatically create a guide to what animals you might see in each month. *(a Naturalist is someone who likes learning about nature, not someone who’s a fan of being naked, that’s a ‘Naturist’… different thing!) Looking for critters in rocky intertidal habitats One of my favourite things to do is going rockpooling, or as we call it over here in California, ‘tidepooling’. Amounting to the same thing, it’s going to a beach that has rocks where the tide covers then uncovers little pools of water at different times of the day. All sorts of fun creatures and life can be found in this ‘rocky intertidal habitat’ A particularly exciting creature that lives here is the Nudibranch, a type of super colourful ‘sea slug’. There are over 3000 species of Nudibranch worldwide. (The word “nudibranch” comes from the Latin nudus, naked, and the Greek βρανχια / brankhia, gills.) ​ They are however quite tricky to find! Even though they are often brightly coloured and interestingly shaped, some of them are very small, and in our part of the world in the Bay Area in California their appearance in our rockpools is seasonal. We see them more often in Summer months, despite the not-as-low tides as in our Winter and Spring seasons. My favourite place to go tidepooling here is Pillar Point in Half Moon bay (at other times of the year more famously known for the surf competition ‘Mavericks’). The rockpools there are rich in species diversity, of varied types and water-coverage habitat zones as well as being relatively accessible. ​ I was rockpooling at Pillar Point recently with my parents and we talked to a lady who remarked that she hadn’t seen any Nudibranchs on her visit this time. I realised that having an idea of what species to find where, and at what time of year is one of the many superpower goals of every budding Naturalist. Using technology and the croudsourced species observations of the iNaturalist community we can shortcut our way to this superpower! Finding nearby animals with iNaturalist We’re going to be getting our information about what animals you can see in Pillar Point using iNaturalist. iNaturalist is a really fun platform that helps connect people to nature and report their findings of life in the outdoors. It is also a community of nature-loving people who help each other identify and confirm those observations. iNaturalist is a project run as a joint initiative by the California Academy of Sciences and the National Geographic Society. I’ve been using iNaturalist for over two years to record and identify plants and animals that I’ve found in the outdoors. I use their iPhone app to upload my pictures, which then uses machine learning algorithms to make an initial guess at what it is I’ve seen. The community is really active, and I often find someone else has verified or updated my species guess pretty soon after posting. This process is great because once an observation has been identified by at least two people it becomes ‘verified’ and is considered research grade. Research grade observations get exported and used by scientists, as well as being indexed by the Global Biodiversity Information Facility, GBIF. ​ iNaturalist has a great API and API explorer, which makes interacting and prototyping using iNaturalist data really fun. For example, if you go to the API explorer and expand the Observations : Search and fetch section and then the GET /observations API, you get a selection of input boxes that allow you to play with options that you can then pass to the API when you click the ‘Try it out’ button. ​ You’ll then get a URL that looks a bit like https://api.inaturalist.org/v1/observations?captive=false &geo=true&verifiable=true&taxon_id=47113&lat=37.495461&lng=-122.499584 &radius=5&order=desc&order_by=created_at which you can call and interrrogate using a programming language of your choice. If you would like to see an all-JavaScript application that uses the iNaturalist API, take a look at OwlsNearMe.com which Simon and I built one weekend earlier this year. It gets your location and shows you all iNaturalist observations of owls near you and lists which species you are likely to see (not adjusted for season). Rapid development using Observable Notebooks We’re going to be using Observable Notebooks to prototype our examples, pulling data down from iNaturalist. I really like using visual notebooks like Observable, they are great for learning and building things quickly. You may be familiar with Jupyter notebooks for Python which is similar but takes a bit of setup to get going - I often use these for prototyping too. Observable is amazing for querying and visualising data with JavaScript and since it is a hosted product it doesn’t require any setup at all. You can follow along and play with this example on my Observable notebook. If you create an account there you can fork my notebook and create your own version of this example. Each ‘notebook’ consists of a page with a column of ‘cells’, similar to what you get in a spreadsheet. A cell can contain Markdown text or JavaScript code and the output of evaluating the cell appears above the code that generated it. There are lots of tutorials out there on Observable Notebooks, I like this code introduction one from Observable (and D3) creator Mike Bostock. Developing your Naturalist superpowers If you have an idea of what plants and critters you might see in a place at the time you visit, you can hone in on what you want to study and train your Naturalist eye to better identify the life around you. For our example, we care about wildlife we can see at Pillar Point, so we need a way of letting the iNaturalist API know which area we are interested in. We could use a latitide, longitude and radius for this, but a rectangular bounding box is a better shape for the reef. We can use this tool to draw the area we want to search within: boundingbox.klokantech.com ​ The tool lets you export the bounding box in several forms using the dropdown at the bottom left under the map givese We are going to use the ‘DublinCore’ format as it’s closest to the format needed by the iNaturalist API. westlimit=-122.50542; southlimit=37.492805; eastlimit=-122.492738; northlimit=37.499811 A quick map primer: The higher the latitude the more north it is The lower the latitude the more south it is Latitude 0 = the equator The higher the longitude the more east it is of Greenwich The lower the longitude the more west it is of Greenwich Longitude 0 = Greenwich In the iNaturalst API we want to use the parameters nelat, nelng, swlat, swlng to create a query that looks inside a bounding box of Pillar Point near Half Moon Bay in California: nelat = highest latitude = north limit = 37.499811 nelng = highest longitude = east limit = -122.492738 swlat = smallest latitude = south limit = 37.492805 swlng = smallest longitude = west limit = 122.50542 As API parameters these look like this: ?nelat=37.499811&nelng=-122.492738&swlat=37.492805&swlng=122.50542 These parameters in this format can be used for most of the iNaturalist API methods. Nudibranch seasonality in Pillar Point We can use the iNaturalist observation_histogram API to get a count of Nudibranch observations per week-of-year across all time and within our Pillar Point bounding box. In addition to the geographic parameters that we just worked out, we are also sending the taxon_id of 47113, which is iNaturalists internal number associated with the Nudibranch taxon. By using this we can get all species which are under the parent ‘Order Nudibranchia’. Another useful piece of naturalist knowledge is understanding the biological classification scheme of Taxanomic Rank - roughly, when a species has a Latin name of two words eg ‘Glaucus Atlanticus’ the first Latin word is the ‘Genus’ like a family name ‘Glaucus’, and the second word identifies that particular species, like a given name ‘Atlanticus’. The two Latin words together indicate a specific species, the term we use colloquially to refer to a type of animal often differs wildly region to region, and sometimes the same common name in two countries can refer to two different species. The common names for the Glaucus Atlanticus (which incidentally is my favourite sea slug) include: sea swallow, blue angel, blue glaucus, blue dragon, blue sea slug and blue ocean slug! Because this gets super confusing, Scientists like using this Latin name format instead. The following piece of code asks the iNaturalist Histogram API to return per-week counts for verified observations of Nudibranchs within our Pillar Point bounding box: pillar_point_counts_per_week = fetch( ""https://api.inaturalist.org/v1/observations/histogram?taxon_id=47113&nelat=37.499811&nelng=-122.492738&swlat=37.492805&swlng=-122.50542&date_field=observed&interval=week_of_year&verifiable=true"" ).then(response => { return response.json(); }) Our next step is to take this data and draw a graph! We’ll be using Vega-Lite for this, which is a fab JavaScript graphing libary that is also easy and fun to use with Observable Notebooks. (Here is a great tutorial on exploring data and drawing graphs with Observable and Vega-Lite) The iNaturalist API returns data that looks like this: { ""total_results"": 53, ""page"": 1, ""per_page"": 53, ""results"": { ""week_of_year"": { ""1"": 136, ""2"": 20, ""3"": 150, ""4"": 65, ""5"": 186, ""6"": 74, ""7"": 47, ""8"": 87, ""9"": 64, ""10"": 56, But for our Vega-Lite graph we need data that looks like this: [{ ""week"": ""01"", ""value"": 136 }, { ""week"": ""02"", ""value"": 20 }, ...] We can convert what we get back from the API to the second format using a loop that iterates over the object keys: objects_to_plot = { let objects = []; Object.keys(pillar_point_counts_per_week.results.week_of_year).map(function(week_index) { objects.push({ week: `Wk ${week_index.toString()}`, observations: pillar_point_counts_per_week.results.week_of_year[week_index] }); }) return objects; } We can then plug this into Vega-Lite to draw us a graph: vegalite({ data: {values: objects_to_plot}, mark: ""bar"", encoding: { x: {field: ""week"", type: ""nominal"", sort: null}, y: {field: ""observations"", type: ""quantitative""} }, width: width * 0.9 }) It’s worth noting that we have a lot of observations of Nudibranchs particularly at Pillar Point due in no small part to the intertidal monitoring research that Alison Young and Rebecca Johnson facilitate for the California Achademy of Sciences. So, what if we want to look for the seasonality of observations of a particular species of adorable sea slug? We want our interface to have a select box with a list of all the species you might find at any time of year. We can do this using the species_counts API to create us an object with the iNaturalist species ID and common & Latin names. pillar_point_nudibranches = { let api_results = await fetch( ""https://api.inaturalist.org/v1/observations/species_counts?taxon_id=47113&nelat=37.499811&nelng=-122.492738&swlat=37.492805&swlng=-122.50542&date_field=observed&verifiable=true"" ).then(r => r.json()) let species_list = api_results.results.map(i => ({ value: i.taxon.id, label: `${i.taxon.preferred_common_name} (${i.taxon.name})` })); return species_list } We can create an interactive select box by importing code from Jeremy Ashkanas’ Observable Notebook: add import {select} from ""@jashkenas/inputs"" to a cell anywhere in our notebook. Observable is magic: like a spreadsheet, the order of the cells doesn’t matter - if one cell is referenced by any other cell then when that cell updates all the other cells refresh themselves. You can also import and reference one notebook from another! viewof select_species = select({ title: ""Which Nudibranch do you want to see seasonality for?"", options: [{value: """", label: ""All the Nudibranchs!""}, ...pillar_point_nudibranches], value: """" }) Then we go back to our old favourite, the histogram API just like before, only this time we are calling it with the value created by our select box ${select_species} as taxon_id instead of the number 47113. pillar_point_counts_per_month_per_species = fetch( `https://api.inaturalist.org/v1/observations/histogram?taxon_id=${select_species}&nelat=37.499811&nelng=-122.492738&swlat=37.492805&swlng=-122.50542&date_field=observed&interval=month_of_year&verifiable=true` ).then(r => r.json()) Now for the fun graph bit! As we did before, we re-format the result of the API into a format compatible with Vega-Lite: objects_to_plot_species_month = { let objects = []; Object.keys(pillar_point_counts_per_month_per_species.results.month_of_year).map(function(month_index) { objects.push({ month: (new Date(2018, (month_index - 1), 1)).toLocaleString(""en"", {month: ""long""}), observations: pillar_point_counts_per_month_per_species.results.month_of_year[month_index] }); }) return objects; } (Note that in the above code we are creating a date object with our specific month in, and using toLocalString() to get the longer English name for the month. Because the JavaScript Date object counts January as 0, we use month_index -1 to get the correct month) And we draw the graph as we did before, only now if you interact with the select box in Observable the graph will dynamically update! vegalite({ data: {values: objects_to_plot_species_month}, mark: ""bar"", encoding: { x: {field: ""month"", type: ""nominal"", sort:null}, y: {field: ""observations"", type: ""quantitative""} }, width: width * 0.9 }) Now we can see when is the best time of year to plan to go tidepooling in Pillar Point if we want to find a specific species of Nudibranch. ​ This tool is great for planning when we to go rockpooling at Pillar Point, but what about if you are going this month and want to pre-train your eye with what to look for in order to impress your friends with your knowledge of Nudibranchs? Well… we can create ourselves a dynamic guide that you can with a list of the species, their photo, name and how many times they have been observed in that month of the year! Our select box this time looks as follows, simpler than before but assigning the month value to the variable selected_month. viewof selected_month = select({ title: ""When do you want to see Nudibranchs?"", options: [ { label: ""Whenever"", value: """" }, { label: ""January"", value: ""1"" }, { label: ""February"", value: ""2"" }, { label: ""March"", value: ""3"" }, { label: ""April"", value: ""4"" }, { label: ""May"", value: ""5"" }, { label: ""June"", value: ""6"" }, { label: ""July"", value: ""7"" }, { label: ""August"", value: ""8"" }, { label: ""September"", value: ""9"" }, { label: ""October"", value: ""10"" }, { label: ""November"", value: ""11"" }, { label: ""December"", value: ""12"" }, ], value: """" }) We then can use the species_counts API to get all the relevant information about which species we can see in month=${selected_month}. We’ll be able to reference this response object and its values later with the variable we just created, eg: all_species_data.results[0].taxon.name. all_species_data = fetch( `https://api.inaturalist.org/v1/observations/species_counts?taxon_id=47113&month=${selected_month}&nelat=37.499811&nelng=-122.492738&swlat=37.492805&swlng=-122.50542&verifiable=true` ).then(r => r.json()) You can render HTML directly in a notebook cell using Observable’s html tagged template literal:

If you go to Pillar Point ${ {"""": """", ""1"":""in January"", ""2"":""in Febrary"", ""3"":""in March"", ""4"":""in April"", ""5"":""in May"", ""6"":""in June"", ""7"":""in July"", ""8"":""in August"", ""9"":""in September"", ""10"":""in October"", ""11"":""in November"", ""12"":""in December"", }[selected_month] } you might see…

${all_species_data.results.map(s => `

${s.taxon.name}

Seen ${s.count} times

`)}
These few lines of HTML are all you need to get this exciting dynamic guide to what Nudibranchs you will see in each month! ​ Play with it yourself in this Observable Notebook. Conclusion I hope by playing with these examples you have an idea of how powerful it can be to prototype using Observable Notebooks and how you can use the incredible crowdsourced community data and APIs from iNaturalist to augment your naturalist skills and impress your friends with your new ‘knowledge of nature’ superpower. Lastly I strongly encourage you to get outside on a low tide to explore your local rocky intertidal habitat, and all the amazing critters that live there. Here is a great introduction video to tidepooling / rockpooling, by Rebecca Johnson and Alison Young from the California Academy of Sciences.",2018,Natalie Downe,nataliedowne,2018-12-18T00:00:00+00:00,https://24ways.org/2018/observable-notebooks-and-inaturalist/,code 47,Developing Robust Deployment Procedures,"Once you have developed your site, how do you make it live on your web hosting? For many years the answer was to log on to your server and upload the files via FTP. Over time most hosts and FTP clients began to support SFTP, ensuring your files were transmitted over a secure connection. The process of deploying a site however remained the same. There are issues with deploying a site in this way. You are essentially transferring files one by one to the server without any real management of that transfer. If the transfer fails for some reason, you may end up with a site that is only half updated. It can then be really difficult to work out what hasn’t been replaced or added, especially where you are updating an existing site. If you are updating some third-party software your update may include files that should be removed, but that may not be obvious to you and you risk leaving outdated files littering your file system. Updating using (S)FTP is a fragile process that leaves you open to problems caused by both connectivity and human error. Is there a better way to do this? You’ll be glad to know that there is. A modern professional deployment workflow should have you moving away from fragile manual file transfers to deployments linked to code committed into source control. The benefits of good practice You may never have experienced any major issues while uploading files over FTP, and good FTP clients can help. However, there are other benefits to moving to modern deployment practices. No surprises when you launch If you are deploying in the way I suggest in this article you should have no surprises when you launch because the code you committed from your local environment should be the same code you deploy – and to staging if you have a staging server. A missing vital file won’t cause things to start throwing errors on updating the live site. Being able to work collaboratively Source control and good deployment practice makes working with your clients and other developers easy. Deploying first to a staging server means you can show your client updates and then push them live. If you subcontract some part of the work, you can give your subcontractor the ability to deploy to staging, leaving you with the final push to launch, once you know you are happy with the work. Having a proper backup of site files with access to them from anywhere The process I will outline requires the use of hosted, external source control. This gives you a backup of your latest commit and the ability to clone those files and start working on them from any machine, wherever you are. Being able to jump back into a site quickly when the client wants a few changes When doing client work it is common for some work to be handed over, then several months might go by without you needing to update the site. If you don’t have a good process in place, just getting back to work on it may take several hours for what could be only a few hours of work in itself. A solid method for getting your local copy up to date and deploying your changes live can cut that set-up time down to a few minutes. The tool chain In the rest of this article I assume that your current practice is to deploy your files over (S)FTP, using an FTP client. You would like to move to a more robust method of deployment, but without blowing apart your workflow and spending all Christmas trying to put it back together again. Therefore I’m selecting the most straightforward tools to get you from A to B. Source control Perhaps you already use some kind of source control for your sites. Today that is likely to be Git but you might also use Subversion or Mercurial. If you are not using any source control at all then I would suggest you choose Git, and that is what I will be working with in this article. When you work with Git, you always have a local repository. This is where your changes are committed. You also have the option to push those changes to a remote repository; for example, GitHub. You may well have come across GitHub as somewhere you can go to download open source code. However, you can also set up private repositories for sites whose code you don’t want to make publicly accessible. A hosted Git repository gives you somewhere to push your commits to and deploy from, so it’s a crucial part of our tool chain. A deployment service Once you have your files pushed to a remote repository, you then need a way to deploy them to your staging environment and live server. This is the job of a deployment service. This service will connect securely to your hosting, and either automatically (or on the click of a button) transfer files from your Git commit to the hosting server. If files need removing, the service should also do this too, so you can be absolutely sure that your various environments are the same. Tools to choose from What follows are not exhaustive lists, but any of these should allow you to deploy your sites without FTP. Hosted Git repositories GitHub Beanstalk Bitbucket Standalone deployment tools Deploy dploy.io FTPloy I’ve listed Beanstalk as a hosted Git repository, though it also includes a bundled deployment tool. Dploy.io is a standalone version of that tool just for deployment. In this tutorial I have chosen two separate services to show how everything fits together, and because you may already be using source control. If you are setting up all of this for the first time then using Beanstalk saves having two accounts – and I can personally recommend them. Putting it all together The steps we are going to work through are: Getting your local site into a local Git repository Pushing the files to a hosted repository Connecting a deployment tool to your web hosting Setting up a deployment Get your local site into a local Git repository Download and install Git for your operating system. Open up a Terminal window and tell Git your name using the following command (use the name you will set up on your hosted repository). > git config --global user.name ""YOUR NAME"" Use the next command to give Git your email address. This should be the address that you will use to sign up for your remote repository. > git config --global user.email ""YOUR EMAIL ADDRESS"" Staying in the command line, change to the directory where you keep your site files. If your files are in /Users/rachel/Sites/mynicewebite you would type: > cd /Users/rachel/Sites/mynicewebsite The next command tells Git that we want to create a new Git repository here. > git init We then add our files: > git add . Then commit the files: > git commit -m “Adding initial files” The bit in quotes after -m is a message describing what you are doing with this commit. It’s important to add something useful here to remind yourself later why you made the changes included in the commit. Your local files are now in a Git repository! However, everything should be just the same as before in terms of working on the files or viewing them in a local web server. The only difference is that you can add and commit changes to this local repository. Want to know more about Git? There are some excellent resources in a range of formats here. Setting up a hosted Git repository I’m going to use Atlassian Bitbucket for my first example as they offer a free hosted and private repository. Create an account on Bitbucket. Then create a new empty repository and give it a name that will identify the repository easily. Click Getting Started and under Command Line select “I have an existing project”. This will give you a set of instructions to run on the command line. The first instruction is just to change into your working directory as we did before. We then add a remote repository, and run two commands to push everything up to Bitbucket. cd /path/to/my/repo git remote add origin https://myuser@bitbucket.org/myname/24ways-tutorial.git git push -u origin --all git push -u origin --tags When you run the push command you will be asked for the password that you set for Bitbucket. Having entered that, you should be able to view the files of your site on Bitbucket by selecting the navigation option Source in the sidebar. You will also be able to see commits. When we initially committed our files locally we added the message “Adding initial files”. If you select Commits from the sidebar you’ll see we have one commit, with the message we set locally. You can imagine how useful this becomes when you can look back and see why you made certain changes to a project that perhaps you haven’t worked on for six months. Before working on your site locally you should run: > git pull in your working directory to make sure you have all of the most up-to-date files. This is especially important if someone else might work on them, or you just use multiple machines. You then make your changes and add any changed or modified files, for example: > git add index.php Commit the change locally: > git commit -m “updated the homepage” Then push it to Bitbucket: > git push origin master If you want to work on your files on a different computer you clone them using the following command: > git clone https://myuser@bitbucket.org/myname/24ways-tutorial.git You then have a copy of your files that is already a Git repository with the Bitbucket repository set up as a remote, so you are all ready to start work. Connecting a deployment tool to your repository and web hosting The next step is deploying files. I have chosen to use a deployment tool called Deploy as it has support for Bitbucket. It does have a monthly charge – but offers a free account for open source projects. Sign up for your account then log in and create your first project. Select Create an empty project. Under Configure Repository Details choose Bitbucket and enter your username and password. If Deploy can connect, it will show you your list of projects. Select the one you want. The next screen is Add New Server and here you need to configure the server that you want to deploy to. You might set up more than one server per project. In an ideal world you would deploy to a staging server for your client preview changes and then deploy once everything is signed off. For now I’ll assume you just want to set up your live site. Give the server a name; I usually use Production for the live web server. Then choose the protocol to connect with. Unless your host really does not support SFTP (which is pretty rare) I would choose that instead of FTP. You now add the same details your host gave you to log in with your SFTP client, including the username and password. The Path on server should be where your files are on the server. When you log in with an SFTP client and you get put in the directory above public_html then you should just be able to add public_html here. Once your server is configured you can deploy. Click Deploy now and choose the server you just set up. Then choose the last commit (which will probably be selected for you) and click Preview deployment. You will then get a preview of which files will change if you run the deployment: the files that will be added and any that will be removed. At the very top of that screen you should see the commit message you entered right back when you initially committed your files locally. If all looks good, run the deployment. You have taken the first steps to a more consistent and robust way of deploying your websites. It might seem like quite a few steps at first, but you will very soon come to realise how much easier deploying a live site is through this process. Your new procedure step by step Edit your files locally as before, testing them through a web server on your own computer. Commit your changes to your local Git repository. Push changes to the remote repository. Log into the deployment service. Hit the Deploy now button. Preview the changes. Run the deployment and then check your live site. Taking it further I have tried to keep things simple in this article because so often, once you start to improve processes, it is easy to get bogged down in all the possible complexities. If you move from deploying with an FTP client to working in the way I have outlined above, you’ve taken a great step forward in creating more robust processes. You can continue to improve your procedures from this point. Staging servers for client preview When we added our server we could have added an additional server to use as a staging server for clients to preview their site on. This is a great use of a cheap VPS server, for example. You can set each client up with a subdomain – clientname.yourcompany.com – and this becomes the place where they can view changes before you deploy them. In that case you might deploy to the staging server, let the client check it out and then go back and deploy the same commit to the live server. Using Git branches As you become more familiar with using Git, and especially if you start working with other people, you might need to start developing using branches. You can then have a staging branch that deploys to staging and a production branch that is always a snapshot of what has been pushed to production. This guide from Beanstalk explains how this works. Automatic deployment to staging I wouldn’t suggest doing automatic deployment to the live site. It’s worth having someone on hand hitting the button and checking that everything worked nicely. If you have configured a staging server, however, you can set it up to deploy the changes each time a commit is pushed to it. If you use Bitbucket and Deploy you would create a deployment hook on Bitbucket to post to a URL on Deploy when a push happens to deploy the code. This can save you a few steps when you are just testing out changes. Even if you have made lots of changes to the staging deployment, the commit that you push live will include them all, so you can do that manually once you are happy with how things look in staging. Further Reading The tutorials from Git Client Tower, already mentioned in this article, are a great place to start if you are new to Git. A presentation from Liam Dempsey showing how to use the GitHub App to connect to Bitbucket Try Git from Code School The Git Workbook a self study guide to Git from Lorna Mitchell Get set up for the new year I love to start the New Year with a clean slate and improved processes. If you are still wrangling files with FTP then this is one thing you could tick off your list to save you time and energy in 2015. Post to the comments if you have suggestions of tools or ideas for ways to enhance this type of set-up for those who have already taken the first steps.",2014,Rachel Andrew,rachelandrew,2014-12-04T00:00:00+00:00,https://24ways.org/2014/developing-robust-deployment-procedures/,process 154,Diagnostic Styling,"We’re all used to using CSS to make our designs live and breathe, but there’s another way to use CSS: to find out where our markup might be choking on missing accessibility features, targetless links, and just plain missing content. Note: the techniques discussed here mostly work in Firefox, Safari, and Opera, but not Internet Explorer. I’ll explain why that’s not really a problem near the end of the article — and no, the reason is not “everyone should just ignore IE anyway”. Basic Diagnostics To pick a simple example, suppose you want to call out all holdover font and center elements in a site. Simple: you just add the following to your styles. font, center {outline: 5px solid red;} You could take it further and add in a nice lime background or some such, but big thick red outlines should suffice. Now you’ll be able to see the offenders wherever as you move through the site. (Of course, if you do this on your public server, everyone else will see the outlines too. So this is probably best done on a development server or local copy of the site.) Not everyone may be familiar with outlines, which were introduced in CSS2, so a word on those before we move on. Outlines are much like borders, except outlines don’t affect layout. Eh? Here’s a comparison. On the left, you have a border. On the right, an outline. The border takes up layout space, pushing other content around and generally being a nuisance. The outline, on the other hand, just draws into quietly into place. In most current browsers, it will overdraw any content already onscreen, and will be overdrawn by any content placed later — which is why it overlaps the images above it, and is overlapped by those below it. Okay, so we can outline deprecated elements like font and center. Is that all? Oh no. Attribution Let’s suppose you also want to find any instances of inline style — that is, use of the style attribute on elements in the markup. This is generally discouraged (outside of HTML e-mails, which I’m not going to get anywhere near), as it’s just another side of the same coin of using font: baking the presentation into the document structure instead of putting it somewhere more manageable. So: *[style], font, center {outline: 5px solid red;} Adding that attribute selector to the rule’s grouped selector means that we’ll now be outlining any element with a style attribute. There’s a lot more that attribute selectors will let use diagnose. For example, we can highlight any images that have empty alt or title text. img[alt=""""] {border: 3px dotted red;} img[title=""""] {outline: 3px dotted fuchsia;} Now, you may wonder why one of these rules calls for a border, and the other for an outline. That’s because I want them to “add together” — that is, if I have an image which possesses both alt and title, and the values of both are empty, then I want it to be doubly marked. See how the middle image there has both red and fuchsia dots running around it? (And am I the only one who sorely misses the actual circular dots drawn by IE5/Mac?) That’s due to its markup, which we can see here in a fragment showing the whole table row. empty title Right, that’s all well and good, but it misses a rather more serious situation: the selector img[alt=""""] won’t match an img element that doesn’t even have an alt attribute. How to tackle this problem? Not a Problem Well, if you want to select something based on a negative, you need a negative selector. img:not([alt]) {border: 5px solid red;} This is really quite a break from the rest of CSS selection, which is all positive: “select anything that has these characteristics”. With :not(), we have the ability to say (in supporting browsers) “select anything that hasn’t these characteristics”. In the above example, only img elements that do not have an alt attribute will be selected. So we expand our list of image-related rules to read: img[alt=""""] {border: 3px dotted red;} img[title=""""] {outline: 3px dotted fuchsia;} img:not([alt]) {border: 5px solid red;} img:not([title]) {outline: 5px solid fuchsia;} With the following results: We could expand this general idea to pick up tables who lack a summary, or have an empty summary attribute. table[summary=""""] {outline: 3px dotted red;} table:not([summary]) {outline: 5px solid red;} When it comes to selecting header cells that lack the proper scope, however, we have a trickier situation. Finding headers with no scope attribute is easy enough, but what about those that have a scope attribute with an incorrect value? In this case, we actually need to pull an on-off maneuver. This has us setting all th elements to have a highlight style, and then turn it off for the elements that meet our criteria. th {border: 2px solid red;} th[scope=""col""], th[scope=""row""] {border: none;} This was necessary because of the way CSS selectors work. For example, consider this: th:not([scope=""col""]), th:not([scope=""row""]) {border: 2px solid red;} That would select…all th elements, regardless of their attrributes. That’s because every th element doesn’t have a scope of col, doesn’t have a scope of row, or doesn’t have either. There’s no escaping this selector o’ doom! This limitation arises because :not() is limited to containing a single “thing” within its parentheses. You can’t, for example, say “select all elements except those that are images which descend from list items”. Reportedly, this limitation was imposed to make browser implementation of :not() easier. Still, we can make good use of :not() in the service of further diagnosing. Calling out links in trouble is a breeze: a[href]:not([title]) {border: 5px solid red;} a[title=""""] {outline: 3px dotted red;} a[href=""#""] {background: lime;} a[href=""""] {background: fuchsia;} Here we have a set that will call our attention to links missing title information, as well as links that have no valid target, whether through a missing URL or a JavaScript-driven page where there are no link fallbacks in the case of missing or disabled JavaScript (href=""#""). And What About IE? As I said at the beginning, much of what I covered here doesn’t work in Internet Explorer, most particularly :not() and outline. (Oh, so basically everything? -Ed.) I can’t do much about the latter. For the former, however, it’s possible to hack your way around the problem by doing some layered on-off stuff. For example, for images, you replace the previously-shown rules with the following: img {border: 5px solid red;} img[alt][title] {border-width: 0;} img[alt] {border-color: fuchsia;} img[alt], img[title] {border-style: double;} img[alt=""""][title], img[alt][title=""""] {border-width: 3px;} img[alt=""""][title=""""] {border-style: dotted;} It won’t have exactly the same set of effects, given the inability to use both borders and outlines, but will still highlight troublesome images. It’s also the case that IE6 and earlier lack support for even attribute selectors, whereas IE7 added pretty much all the attribute selector types there are, so the previous code block won’t have any effect previous to IE7. In a broader sense, though, these kinds of styles probably aren’t going to be used in the wild, as it were. Diagnostic styles are something only you see as you work on a site, so you can make sure to use a browser that supports outlines and :not() when you’re diagnosing. The fact that IE users won’t see these styles is irrelevant since users of any browser probably won’t be seeing these styles. Personally, I always develop in Firefox anyway, thanks to its ability to become a full-featured IDE through the addition of extensions like Firebug and the Web Developer Toolbar. Yeah, About That… It’s true that much of what I describe in this article is available in the WDT. I feel there are two advantages to writing your own set of diagnostic styles. When you write your own styles, you can define exactly what the visual results will be, and how they will interact. The WDT doesn’t let you make its outlines thicker or change their colors. You can combine a bunch of diagnostics into a single set of rules and add it to your site’s style sheet during the diagnostic portion, thus ensuring they persist as you surf around. This can be done in the WDT, but it isn’t as easy (and, at least for me, not as reliable). It’s also true that a markup validator will catch many of the errors I mentioned, such as missing alt and summary attributes. For some, that’s sufficient. But it won’t catch everything diagnostic styles can, like empty alt values or untargeted links, which are perfectly valid, syntactically speaking. Diagnosis Complete? I hope this has been a fun look at the concept of diagnostic styling as well as a quick introduction into possibly new concepts like :not() and outlines. This isn’t all there is to say, of course: there is plenty more that could be added to a diagnostic style sheet. And everyone’s diagnostics will be different, tuned to meet each person’s unique situation. Mostly, though, I hope this small exploration triggers some creative thinking about the use of CSS to do more than just lay out pages and colorize text. Given the familiarity we acquire with CSS, it only makes sense to use it wherever it might be useful, and setting up visible diagnostic flags is just one more place for it to help us.",2007,Eric Meyer,ericmeyer,2007-12-20T00:00:00+00:00,https://24ways.org/2007/diagnostic-styling/,process 89,"Direction, Distance and Destinations","With all these new smartphones in the hands of lost and confused owners, we need a better way to represent distances and directions to destinations. The immediate examples that jump to mind are augmented reality apps which let you see another world through your phone’s camera. While this is interesting, there is a simpler way: letting people know how far away they are and if they are getting warmer or colder. In the app world, you can easily tap into the phone’s array of sensors such as the GPS and compass, but what people rarely know is that you can do the same with HTML. The native versus web app debate will never subside, but at least we can show you how to replicate some of the functionality progressively in HTML and JavaScript. In this tutorial, we’ll walk through how to create a simple webpage listing distances and directions of a few popular locations around the world. We’ll use JavaScript to access the device’s geolocation API and also attempt to access the compass to get a heading. Both of these APIs are documented, to be included in the W3C geolocation API specification, and can be used on both desktop and mobile devices today. To get started, we need a list of a few locations around the world. I have chosen the highest mountain peak on each continent so you can see a diverse set of distances and directions. Mountain °Latitude °Longitude Kilimanjaro -3.075833 37.353333 Vinson Massif -78.525483 -85.617147 Puncak Jaya -4.078889 137.158333 Everest 27.988056 86.925278 Elbrus 43.355 42.439167 Mount McKinley 63.0695 -151.0074 Aconcagua -32.653431 -70.011083 Source: Wikipedia We can put those into an HTML list to be styled and accessed by JavaScript to create some distance and directions calculations. The next thing we need to do is check to see if the browser and operating system have geolocation support. To do this we test to see if the function is available or not using a single JavaScript if statement. The if statement will be false if geolocation support is not present, and then it is up to you to do something else instead as a fallback. For this example, we’ll do nothing since our page should work as is and only get progressively better if more functionality is available. The if statement will be true if there is support and therefore will continue inside the curly brackets to try to get the location. This should prompt the reader to accept or deny the request to get their location. If they say no, the second function callback is processed, in this case a function called geo_error; whereas if the location is available, it fires the geo_success function callback. The function geo_error(){ } isn’t that exciting. You can handle this in any way you see fit. The success function is more interesting. We get a position object passed into the function which contains a series of exciting attributes, namely the latitude and longitude of the device’s current location. function geo_success(position){ gLat = position.coords.latitude; gLon = position.coords.longitude; } Now, in the variables gLat and gLon we have the user’s approximate geographical position. We can use this information to start to calculate some distances between where they are and all the destinations. At the time of writing, you can also get position.coords.heading, but on Windows and iOS devices this returned NULL. In the future, if and when this is supported, this is also where you can easily grab the compass information. Inside the geo_success function, we want to loop through the HTML to get all of the mountain peaks’ latitudes and longitudes and compute the distance. ... $('.geo').each(function(){ // Get the lat/lon from the HTML tLat = $(this).find('.lat').html() tLon = $(this).find('.lon').html() // compute the distances between the current location and this points location dist = distance(tLat,tLon,gLat,gLon); // set the return values into something useful d = parseInt(dist[0]*10)/10; a = parseFloat(dist[1]); // display the value in the HTML and style the arrow $(this).find('.distance').html(d+' km away'); $(this).find('.direction').css('-webkit-transform','rotate(-' + a + 'deg)'); // store the arc for later use if compass is available $(this).attr('data-arc',a); } In the variable d we have the distance between the current location and the location of the mountain peak based on the Haversine Formula. The variable a is the arc, which has a value from 0 to 359.99. This will be useful later if we have compass support. Given these two values we have a distance and a heading to style the HTML. The next thing we want to do is check to see if the device has a compass and then get access to the the current heading. As we’ll see, there are several ways to do this, some of which work on certain devices but not others. The W3C geolocation spec says that, along with the coordinates, there are several other attributes: accuracy; altitude; and heading. Heading is the direction to true north, which is different than magnetic north! WebKit and Windows return NULL for the heading value, but WebKit has an experimental method to fetch the heading. If you get into accessing these sensors, you’ll have to try to catch a few of these methods to finally get a value. Assuming you do, we can move on to the more interesting display opportunities. In an ideal world, this would succeed and set a variable we’ll call compassHeading to get a value between 0 and 359.99 degrees. Now we know which direction north is, we also know the direction relative to north of the path to our destination, so we can can subtract the two values to get an arrow to display on the screen. But we’re not finished yet: we also need to get the device’s orientation (landscape or portrait) and subtract the correct amount from the angle for the arrow. Once we have a value, we can use CSS to rotate the arrow the correct number of degrees. -webkit-transform: rotate(-180deg) Not all devices support a standard way to access compass information, so in the meantime we need to use a work around. On iOS, you can use the experimental event method e.webkitCompassHeading. We want the compass to update in real time as the device is moved around, so we’ll put this inside an event listener. window.addEventListener('deviceorientation', function(e) { // Loop through all the locations on the page $('.geo').each(function(){ // get the arc value from north we computed and stored earlier destination_arc = parseInt($(this).attr('data-arc')) compassHeading = e.webkitCompassHeading + window.orientation + destination_arc; // find the arrow element and rotate it accordingly $(this).find('.direction').css('-webkit-transform','rotate(-' + compassHeading + 'deg)'); } } As the device is rotated, the compass arrow will constantly be updated. If you want to see an example, you can have a look at this page which shows the distances to all the peaks on each continent. With progressive enhancement, we slowly layer on additional functionality as we go. The reader will first see the list of locations with a latitude and longitude. If the device is capable and permissions allow, it will then compute the distance. If a compass is available, with the correct permissions it will then add the final layer which is direction. You should consider this code a stub for your projects. If you are making a hyperlocal webpage with restaurant locations, for example, then consider adding these features. Knowing not only how far away a place is, but also the direction can be hugely important, and since the compass is always active, it acts as a guide to the location. Future developments Improvements to this could include setting a timer and recalling the navigator.geolocation.getCurrentPosition() function and updating the distances. I chose very distant mountains so kilometres made sense, but you can divide again by 1,000 to convert to metres if you are dealing with much nearer places. Walking or driving would change the distances so the ability to refresh would be important. It is outside the scope of this article, but if you manage to get this HTML to work offline, then you can make a nice web app which sits on your devices’ homescreens and works even without an internet connection. This could be ideal for travellers in an unknown city looking for your destination. Just with offline storage, base64 encoding and data URIs, it is possible to embed plenty of design and functionality into a small offline webpage. Now you know how to use JavaScript to look up a destination’s location and figure out the distance and direction – never get lost again.",2012,Brian Suda,briansuda,2012-12-19T00:00:00+00:00,https://24ways.org/2012/direction-distance-and-destinations/,code 288,Displaying Icons with Fonts and Data- Attributes,"Traditionally, bitmap formats such as PNG have been the standard way of delivering iconography on websites. They’re quick and easy, and it also ensures they’re as pixel crisp as possible. Bitmaps have two drawbacks, however: multiple HTTP requests, affecting the page’s loading performance; and a lack of scalability, noticeable when the page is zoomed or viewed on a screen with a high pixel density, such as the iPhone 4 and 4S. The requests problem is normally solved by using CSS sprites, combining the icon set into one (physically) large image file and showing the relevant portion via background-position. While this works well, it can get a bit fiddly to specify all the positions. In particular, scalability is still an issue. A vector-based format such as SVG sounds ideal to solve this, but browser support is still patchy. The rise and adoption of web fonts have given us another alternative. By their very nature, they’re not only scalable, but resolution-independent too. No need to specify higher resolution graphics for high resolution screens! That’s not all though: Browser support: Unlike a lot of new shiny techniques, they have been supported by Internet Explorer since version 4, and, of course, by all modern browsers. We do need several different formats, however! Design on the fly: The font contains the basic graphic, which can then be coloured easily with CSS – changing colours for themes or :hover and :focus styles is done with one line of CSS, rather than requiring a new graphic. You can also use CSS3 properties such as text-shadow to add further effects. Using -webkit-background-clip: text;, it’s possible to use gradient and inset shadow effects, although this creates a bitmap mask which spoils the scalability. Small file size: specially designed icon fonts, such as Drew Wilson’s Pictos font, can be as little as 12Kb for the .woff font. This is because they contain fewer characters than a fully fledged font. You can see Pictos being used in the wild on sites like Garrett Murray’s Maniacal Rage. As with all formats though, it’s not without its disadvantages: Icons can only be rendered in monochrome or with a gradient fill in browsers that are capable of rendering CSS3 gradients. Specific parts of the icon can’t be a different colour. It’s only appropriate when there is an accompanying text to provide meaning. This can be alleviated by wrapping the text label in a tag (I like to use rather than , due to the fact that it’s smaller and isn’ t being used elsewhere) and then hiding it from view with text-indent:-999em. Creating an icon font can be a complex and time-consuming process. While font editors can carry out hinting automatically, the best results are achieved manually. Unless you’re adept at creating your own fonts, you’re restricted to what is available in the font. However, fonts like Pictos will cover the most common needs, and icons are most effective when they’re using familiar conventions. The main complaint about using fonts for icons is that it can mean adding a meaningless character to our markup. The good news is that we can overcome this by using one of two methods – CSS generated content or the data-icon attribute – in combination with the :before and :after pseudo-selectors, to keep our markup minimal and meaningful. Our simple markup looks like this: View Basket Note the multiple class attributes. Next, we’ll import the Pictos font using the @font-face web fonts property in CSS: @font-face { font-family: 'Pictos'; src: url('pictos-web.eot'); src: local('☺'), url('pictos-web.woff') format('woff'), url('pictos-web.ttf') format('truetype'), url('pictos-web.svg#webfontIyfZbseF') format('svg'); } This rather complicated looking set of rules is (at the time of writing) the most bulletproof way of ensuring as many browsers as possible load the font we want. We’ll now use the content property applied to the :before pseudo-class selector to generate our icon. Once again, we’ll use those multiple class attribute values to set common icon styles, then specific styles for .basket. This helps us avoid repeating styles: .icon { font-family: 'Pictos'; font-size: 22px: } .basket:before { content: ""$""; } What does the :before pseudo-class do? It generates the dollar character in a browser, even when it’s not present in the markup. Using the generated content approach means our markup stays simple, but we’ll need a new line of CSS, defining what letter to apply to each class attribute for every icon we add. data-icon is a new alternative approach that uses the HTML5 data- attribute in combination with CSS attribute selectors. This new attribute lets us add our own metadata to elements, as long as its prefixed by data- and doesn’t contain any uppercase letters. In this case, we want to use it to provide the letter value for the icon. Look closely at this markup and you’ll see the data-icon attribute. View Basket We could add others, in fact as many as we like. Favourites History Location Then, we need just one CSS attribute selector to style all our icons in one go: .icon:before { content: attr(data-icon); /* Insert your fancy colours here */ } By placing our custom attribute data-icon in the selector in this way, we can enable CSS to read the value of that attribute and display it before the element (in this case, the anchor tag). It saves writing a lot of CSS rules. I can imagine that some may not like the extra attribute, but it does keep it out of the actual content – generated or not. This could be used for all manner of tasks, including a media player and large simple illustrations. See the demo for live examples. Go ahead and zoom the page, and the icons will be crisp, with the exception of the examples that use -webkit-background-clip: text as mentioned earlier. Finally, it’s worth pointing out that with both generated content and the data-icon method, the letter will be announced to people using screen readers. For example, with the shopping basket icon above, the reader will say “dollar sign view basket”. As accessibility issues go, it’s not exactly the worst, but could be confusing. You would need to decide whether this method is appropriate for the audience. Despite the disadvantages, icon fonts have huge potential.",2011,Jon Hicks,jonhicks,2011-12-12T00:00:00+00:00,https://24ways.org/2011/displaying-icons-with-fonts-and-data-attributes/,code 226,Documentation-Driven Design for APIs,"Documentation is like gift wrapping. It seems like superfluous fluff, but your family tends to be rather disappointed when their presents arrive in supermarket carrier bags, so you have to feign some sort of attempt at making your gift look enticing. Documentation doesn’t have to be all hard work and sellotaping yourself to a table – you can make it useful and relevant. Documentation gets a pretty rough deal. It tends to get left until the end of a project, when some poor developer is assigned the ‘document project’ ticket and wades through each feature of Whizzy New API 3.0 and needs to recall exactly what each method is meant to do. That’s assuming any time is left for documentation at all. The more common outcome resembles last minute homework scribbled on a post-it note, where just the bare bones of what’s available are put out for your users, and you hope that you’ll spot the inconsistencies and mistakes before they do. Wouldn’t it be nicer for everyone if you could make documentation not only outstanding for your users, but also a valuable tool for your development team – so much so that you couldn’t imagine writing a line of code before you’d documented it? Documentation needs to have three main features: It should have total coverage and document all the features of your project. Private methods should be documented for your developers, and public features need to be available to your users. It should be consistent – a user should know what to expect from your documentation, and terminology should be accurate to your language. It should be current – and that means staying accurate as new versions of your code base are released. But you can also get these bonuses: Act as a suggested specification – a guide that will aid a developer in making something consistent and usable. It can test your API quality. It can enhance the communication skills within your development team. So how do we get our documentation to be rich and full of features, instead of a little worn out like Boxing Day leftovers? Write your documentation first When I say first, I mean first. Not after you’ve started writing the code. Not even after you’ve started writing your unit tests. First. You may or may not have been provided with a decent specification, but the first job should be to turn your requirements for a feature into documentation. It works best when it takes the form of in-code comments. It works even better when your in-code comments take a standard documentation format that you can later use to generate published documentation for your users. This has the benefit of immediately making your docs as version controlled as your code-base, and it saves having to rewrite, copy or otherwise harass your docs into something legible later on. Almost all languages have a self-documentation format these days. My choice of format for JavaScript is JSDocToolkit, and the sort of things I look for are the ability to specify private and public methods, full options object statements (opts as Opts only is a no-no), and the ability to include good examples. So, our example for today will be a new festive feature for a JavaScript API. We’ve been asked to specify a sled for Santa to get around the world to give out toys: Santa needs to be able to travel around the world in one night to deliver toys to children, and he’ll need some reindeer to pull his sled. As documentation, it would look like: /** @name Sled @extends Vehicle @constructor @description Create a new sled to send Santa around the world to deliver toys to good kids. @param {Object} [opts] Options @param {number} [opts.capacity='50'] Set the capacity of the sled @param {string} [opts.pilot='santa'] The pilot of the sled. @example // Create a sled and specify some reindeer. new Sled().reindeer(['Dasher', 'Dancer', 'Prancer', 'Vixen', 'Comet', 'Cupid']); */ By breaking it down as documentation, you can, for example, hand this over to another developer without the need to explain the feature in much depth, and they’ll develop something that has to match this piece of documentation. It specifies everything that is important to this feature – its default values and types, and where it inherits other features from. We know that we need to specify some way of setting reindeer to pull the sled and also some toys to give, and so we can quickly specify extra methods for the sled: /* @name vehicle.Sled#reindeer @function @description Set the reindeer that will pull Santa's sled. @param {string[]} reindeer A list of the reindeer. @example // specifying some reindeer Sled().reindeer(['Dasher', 'Dancer', 'Rudolph', 'Vixen']); */ /* @name vehicle.Sled#toys @function @description Add a list of toys and recipients to the Sled. @param {Object[]} toys A list of toys and who will receive them. @example // Adding toys to the sled Sled().toys([ {name:'Brian', toy:'Fire Engine'}, {name:'Drew', toy:'Roller-skates'}, {name:'Anna', toy:'Play-doh'}, ... ]); */ Job done! You’ve got a specification to share with your team and something useful for your users in the form of full examples, and you didn’t even have to open another text editor. Use your documentation to share knowledge Documentation isn’t just for users. It’s also used by internal developers to explain what they’ve written and how it works. This is especially valuable where the team is large or the code-base sprawling. So, returning to our example, the next step would be to share with the rest of the team (or at least a selection of the team if yours is large) what the documentation looks like. This is useful for two main reasons: They can see if they understand what the documentation says the feature will do. It’s best if they haven’t seen the requirement before. If your fellow developers can’t work out what ‘MagicMethodX’ is going to return from the docs, neither can your users. They can check that the feature accomplishes everything that they expect to, and that it’s consistent with the rest of the functionality. On previous projects, we’ve taken to referring to this stage of the development process as the ‘bun fight’. It’s a chance for everyone to have an honest say and throw a few pies without actually causing anyone to have to rewrite any code. If you can identify at this stage that a feature is over-complicated, lacking or just plain useless, you’ll all be much happier to throw out a few lines of documentation than you may have been to throw out a partial, or even complete, piece of functionality. Documentation has your back The final benefit to working in this way is that your documentation not only remains accurate, it’s always as accurate as your latest release. It can’t fall behind. You can increase the likelihood that your docs will remain up to date by unit testing your examples. Returning to the previous example, we can add a QUnit unit test to the expected output with ease during the build process – we know exactly how the code will look and, with the @example tag, we can identify easily where to find the bits that need testing. If it’s tested it’ll definitely work as you expect it to when a user copy and pastes it. You’re ensuring quality from idea to implementation. As an extra bauble, the best thing about a system like JSDocToolkit is that it’ll take your inline comments and turn them into beautiful sites, as good systems will allow for customised output templates. You’ll be producing full-featured sites for your projects and plugins with almost no extra effort, but all the benefits.",2010,Frances Berriman,francesberriman,2010-12-11T00:00:00+00:00,https://24ways.org/2010/documentation-driven-design-for-apis/,process 188,Don't Lose Your :focus,"For many web designers, accessibility conjures up images of blind users with screenreaders, and the difficulties in making sites accessible to this particular audience. Of course, accessibility covers a wide range of situations that go beyond the extreme example of screenreader users. And while it’s true that making a complex site accessible can often be a daunting prospect, there are also many small things that don’t take anything more than a bit of judicious planning, are very easy to test (without having to buy expensive assistive technology), and can make all the difference to certain user groups. In this short article we’ll focus on keyboard accessibility and how careless use of CSS can potentially make your sites completely unusable. Keyboard Access Users who for whatever reason can’t use a mouse will employ a keyboard (or keyboard-like custom interface) to navigate around web pages. By default, they will use TAB and SHIFT + TAB to move from one focusable element (links, form controls and area) of a page to the next. Note: in OS X, you’ll first need to turn on full keyboard access under System Preferences > Keyboard and Mouse > Keyboard Shortcuts. Safari under Windows needs to have the option Press Tab to highlight each item on a webpage in Preferences > Advanced enabled. Opera is the odd one out, as it has a variety of keyboard navigation options – the most relevant here being spatial navigation via Shift+Down, Shift+Up, Shift+Left, and Shift+Right). But I Don’t Like Your Dotted Lines… To show users where they are within a page, browsers place an outline around the element that currently has focus. The “problem” with these default outlines is that some browsers (Internet Explorer and Firefox) also display them when a user clicks on a focusable element with the mouse. Particularly on sites that make extensive use of image replacement on links with “off left” techniques this can create very unsightly outlines that stretch from the replaced element all the way to the left edge of the browser. Outline bleeding off to the left (image-replacement example from carsonified.com) There is a trivial workaround to prevent outlines from “spilling over” by adding a simple overflow:hidden, which keeps the outline in check around the clickable portion of the image-replaced element itself. Outline tamed with overflow:hidden But for many designers, even this is not enough. As a final solution, many actively suppress outlines altogether in their stylesheets. Controversially, even Eric Meyer’s popular reset.css – an otherwise excellent set of styles that levels the playing field of varying browser defaults – suppresses outlines. html, body, div, span, applet, object, iframe ... { ... outline: 0; ... } /* remember to define focus styles! */ :focus { outline: 0; } Yes, in his explanation (and in the CSS itself) Eric does remind designers to define relevant styles for :focus… but judging by the number of sites that seem to ignore this (and often remove the related comment from the stylesheet altogether), the message doesn’t seem to have sunk in. Anyway… hurrah! No more unsightly dotted lines on our lovely design. But what about keyboard users? Although technically they can still TAB from one element to the next, they now get no default cue as to where they are within the page (one notable exception here is Opera, where the outline is displayed regardless of stylesheets)… and if they’re Safari users, they won’t even get an indication of a link’s target in the status bar, like they would if they hovered over it with the mouse. Only Suppress outline For Mouse Users Is there a way to allow users navigating with the keyboard to retain the standard outline behaviour they’ve come to expect from their browser, while also ensuring that it doesn’t show display for mouse users? Testing some convoluted style combinations After playing with various approaches (see Better CSS outline suppression for more details), the most elegant solution also seemed to be the simplest: don’t remove the outline on :focus, do it on :active instead – after all, :active is the dynamic pseudo-class that deals explicitly with the styles that should be applied when a focusable element is clicked or otherwise activated. a:active { outline: none; } The only minor issues with this method: if a user activates a link and then uses the browser’s back button, the outline becomes visible. Oh, and old versions of Internet Explorer notoriously get confused by the exact meaning of :focus, :hover and :active, so this method fails in IE6 and below. Personally, I can live with both of these. Note: at the last minute before submitting this article, I discovered a fatal flaw in my test. It appears that outline still manages to appear in the time between activating a link and the link target loading (which in hindsight is logical – after activation, the link does indeed receive focus). As my test page only used in-page links, this issue never came up before. The slightly less elegant solution is to also suppress the outline on :hover. a:hover, a:active { outline: none; } In Conclusion Of course, many web designers may argue that they know what’s best, even for their keyboard-using audience. Maybe they’ve removed the default outline and are instead providing some carefully designed :focus styles. If they know for sure that these custom styles are indeed a reliable alternative for their users, more power to them… but, at the risk of sounding like Jakob “blue underlined links” Nielsen, I’d still argue that sometimes the default browser behaviours are best left alone. Complemented, yes (and if you’re already defining some fancy styles for :hover, by all means feel free to also make them display on :focus)… but not suppressed.",2009,Patrick Lauke,patricklauke,2009-12-09T00:00:00+00:00,https://24ways.org/2009/dont-lose-your-focus/,code 326,Don't be eval(),"JavaScript is an interpreted language, and like so many of its peers it includes the all powerful eval() function. eval() takes a string and executes it as if it were regular JavaScript code. It’s incredibly powerful and incredibly easy to abuse in ways that make your code slower and harder to maintain. As a general rule, if you’re using eval() there’s probably something wrong with your design. Common mistakes Here’s the classic misuse of eval(). You have a JavaScript object, foo, and you want to access a property on it – but you don’t know the name of the property until runtime. Here’s how NOT to do it: var property = 'bar'; var value = eval('foo.' + property); Yes it will work, but every time that piece of code runs JavaScript will have to kick back in to interpreter mode, slowing down your app. It’s also dirt ugly. Here’s the right way of doing the above: var property = 'bar'; var value = foo[property]; In JavaScript, square brackets act as an alternative to lookups using a dot. The only difference is that square bracket syntax expects a string. Security issues In any programming language you should be extremely cautious of executing code from an untrusted source. The same is true for JavaScript – you should be extremely cautious of running eval() against any code that may have been tampered with – for example, strings taken from the page query string. Executing untrusted code can leave you vulnerable to cross-site scripting attacks. What’s it good for? Some programmers say that eval() is B.A.D. – Broken As Designed – and should be removed from the language. However, there are some places in which it can dramatically simplify your code. A great example is for use with XMLHttpRequest, a component of the set of tools more popularly known as Ajax. XMLHttpRequest lets you make a call back to the server from JavaScript without refreshing the whole page. A simple way of using this is to have the server return JavaScript code which is then passed to eval(). Here is a simple function for doing exactly that – it takes the URL to some JavaScript code (or a server-side script that produces JavaScript) and loads and executes that code using XMLHttpRequest and eval(). function evalRequest(url) { var xmlhttp = new XMLHttpRequest(); xmlhttp.onreadystatechange = function() { if (xmlhttp.readyState==4 && xmlhttp.status==200) { eval(xmlhttp.responseText); } } xmlhttp.open(""GET"", url, true); xmlhttp.send(null); } If you want this to work with Internet Explorer you’ll need to include this compatibility patch.",2005,Simon Willison,simonwillison,2005-12-07T00:00:00+00:00,https://24ways.org/2005/dont-be-eval/,code 40,Don’t Push Through the Pain,"In 2004, I lost my web career. In a single day, it was gone. I was in too much pain to use a keyboard, a Wacom tablet (I couldn’t even click the pen), or a trackball. Switching my mouse to use my left (non-dominant) hand only helped a bit; then that hand went, too. I tried all the easy-to-find equipment out there, except for expensive gizmos with foot pedals. I had tingling in my fingers—which, when I was away from the computer, would rhythmically move as if some other being controlled them. I worried about Parkinson’s because the movements were so dramatic. Pen on paper was painful. Finally, I discovered one day that I couldn’t even turn a doorknob. The only highlight was that I couldn’t dust, scrub, or vacuum. We were forced to hire someone to come in once a week for an hour to whip through the house. You can imagine my disappointment. My injuries had gradually slithered into my life without notice. I’d occasionally have sore elbows, or my wrist might ache for a day, or my shoulders feel tight. But nothing to keyboard home about. That’s the critical bit of news. One day, you’re pretty fine. The next day, you don’t have your job—or any job that requires the use of your hands and wrists. I had to walk away from the computer for over four months—and partially for several months more. That’s right: no income. If I hadn’t found a gifted massage therapist, the right book of stretches, the equipment I should have been using all along, and learned how to pay attention to my body—even just a little bit more—I quite possibly wouldn’t be writing this article today. I wouldn’t be writing anything, anywhere. Most of us have heard of (and even claimed to have read all of) Mihaly Csikszentmihalyi, author of Flow: The Psychology of Optimal Experience, who describes the state of flow—the place our minds go when we are fully engaged and in our element. This lovely state of highly focused activity is deeply satisfying, often creative, and quite familiar to many of us on the web who just can’t quit until the copy sings or the code is untangled or we get our highest score yet in Angry Birds. Our minds may enter that flow, but too often as our brains take flight, all else recedes. And we leave something very important behind. Our bodies. My body wasn’t made to make the same minute movements thousands of times a day, most days of the year, for decades, and neither was yours. The wear and tear sneaks up on you, especially if you’re the obsessive perfectionist that we all pretend not to be. Oh? You’re not obsessed? I wasn’t like this all the time, but I remember sitting across from my husband, eating dinner, and I didn’t hear a word he said. I’d left my brain upstairs in my office, where it was wrestling in a death match with the box model or, God help us all, IE 5.2. I was a writer, too, and I was having my first inkling that I was a content strategist. Work was exciting. I could sit up late, in the flow, fingers flying at warp speed. I could sit until those wretched birds outside mocked me with their damn, cheerful “Hurray, it’s morning!” songs. Suddenly, while, say, washing dishes, the one magical phrase that captured the essence of a voice or idea would pop up, and I would have mowed down small animals and toddlers to get to my computer and hammer out that website or article, to capture that thought before it escaped. Note my use of the word hammer. Sound at all familiar? But where was my body during my work? Jaw jutting forward to see the screen, feet oddly positioned—and then left in place like chunks of marble—back unsupported, fingers pounding the keys, wrists and arms permanently twisted in unnatural angles that we thought were natural. And clicking. Clicking, clicking, clicking that mouse. Thumbing tiny keyboards on phones. A lethal little gesture for tiny little tendons. Though I was fine from, say 1997 to 2004, by the end of 2004 this behavior culminated in disaster. I had repetitive stress injuries, aka repetitive motion injuries. As the Apple site says, “A brief exposure to these conditions would not cause harm. But a prolonged exposure may, in some people, result in reduced ability to function.” I’ll say. I frantically turned to people on lists and forums. “Try a track ball.” Already did that. “Try a tablet.” Worse. One person wrote, “I still come here once in a while and can type a couple sentences, but I’ve permanently got thoracic outlet syndrome and I’ll never work again.” Oh, beauteous web, oh, long-distance friends, farewell. The Wrist Bone’s Connected to the Brain Bone That variation on the old song tells part of the story. Most people (and many of their physicians) believe that tingling fingers and aching wrists MUST be carpel tunnel syndrome. Nope. If your neck juts forward, it tenses and stays tense the entire time you work in that position. Remember how your muscles felt after holding a landline phone with your neck tilted to one side for a long client meeting? Regrettable. Tensing your shoulders because your chair’s not designed properly puts you at risk for thoracic outlet syndrome, a career-killer if ever there was one. The nerves and tendons in your neck and shoulder refer down your arms, and muscles swell around nerves, causing pain and dysfunction. Your elbows have a tendon that is especially vulnerable to repetitive movements (think tennis elbow). Your wrists are performing something akin to a circus act with one thousand shows a day. So, all the fine tendons and ligaments in your fingers have problems that may not start at your wrists at all. Though some people truly do have carpal tunnel syndrome, my finger and wrist problems weren’t solved by heavily massaging my fingers (though, that was helpful, too) or my wrists. They were fixed by work on my neck, upper back, shoulders, arms, and elbows. This explains why many people have surgery for carpal tunnel syndrome and just months later say, “What?! How can I possibly have it again? I had an operation!” Well, fellow buckaroo, you may never have had carpel tunnel syndrome. You may have had—or perhaps will have—one long disaster area from your neck to your fingertips. How to Crawl Back Before trying extreme measures, you may be able to function again even if you feel hopeless. I managed to heal, and so have others, but I’ll always be at risk. As Jen Simmons, of The Web Ahead podcast and other projects told me, “It took a long time to injure myself. It took a long time to get back to where I was. My right arm between my elbow and wrist would start aching intermittently. Eventually, my arm even ached at night. I started each day with yesterday’s pain.” Simple measures, used consistently, helped her back. 1. Massage therapy I don’t remember what the rest of the world is like, but in Portland, Oregon, we have more than one massage therapy college. (Of course we do.) I saw a former teacher at the most respected school. This is not your “It was all so soothing. Why, I fell asleep!” massage. This is “Holy crap, he’s grinding his elbow into my armpit!“ massage therapy, with the emphasis on therapy. I owe him everything. Make sure you have someone who really knows what they’re doing. Get many referrals. Try a question, “Does my psoas muscle affect my back?” If they can’t answer it, flee. Regularly see the one you choose and after a while, depending on how injured you are, you may be able to taper off. 2. Change your equipment You may need to be hands-on with several pieces of equipment before you find the ones that don’t cause more pain. Many companies have restocking fees, charges to ship the equipment you want to return, and other retail atrocities. Always be sure to ask what the return policies are at any company before purchasing. Mice You may have more success than I did with equipment such as the Wacom tablet. Mine came with a pen, and it hurt to repetitively click it. Trackballs are another option but, for many, they are better at prevention than recovery. But let’s get to the really effective stuff. One of the biggest sources of pain is using your mouse. One major reason is that your hand and wrist are in a perpetually unnatural position and you’re also moving your arm quite a bit. Each time you move the mouse, it is placing stress on your neck, shoulders and arms, because you need to lift them slightly in order to move the mouse and you need to angle your wrist. You may also be too injured to use the trackpad all the time, and this mouse, the vertical mouse is a dandy preventative measure, too. Shaking up your patterns is a wise move. I have long fingers, not especially thin, yet the small size works best for me. (They have larger choices available.) What?! A sideways mouse? Yep. All the weight of your hand will be resting on it in the handshake position. Your forearms aren’t constantly twisting over hill and dale. You aren’t using any muscles in your wrist or hand. They are relaxing. You’ll adapt in a day, and oh, oh, what a relief it is. Keyboards I really liked doing business with the people at Kinesis-Ergo. (I’m not affiliated with them in any way.) They have the vertical mouse and a number of keyboards. The one that felt the most natural to me, and, once again, it only takes a day to adapt, is the Freestyle2 for the Mac. They have several options. I kept the keyboard halves attached to each other at first, and then spread them apart a little more. I recommend choosing one that slants and can separate. You can adjust the angle. For a little extra, they’ll make sure it’s all set up and ready to go for you. I’m guessing that some Googling will find you similar equipment, wherever you live. Warning: if you use the ergonomic keyboards, you may have fewer USB ports. The laptop will be too far away to see unless you find a satisfactory setup using a stand. This is the perfect excuse for purchasing a humongous display. You may not look cool while jetting coast to coast in your skinny jeans and what appears to be the old-time orthopedic shoe version of computing gear. But once you have rested and used many of these suggestions consistently, you may be able to use your laptop or other device in all its lovely sleekness during the trip. Other doohickies The Kinesis site and The Human Solution have a wide selection of ergonomic products: standing desks, ergonomically correct chairs, and, yes, even things with foot pedals. Explore! 3. Stop clicking, at least for a while Use keyboard shortcuts, but use them slowly. This is not the time to show off your skillz. You’ll be sort of like a recovering alcoholic, in that you’ll be a recovering repetitive stress survivor for the rest of your life, once you really injure yourself. Always be vigilant. There’s also a bit of software sold by The Human Solution and other places, and it was my salvation. It’s called the McNib for Macs, and the Nib for PCs. (I’ve only used the McNib.) It’s for click-free mousing. I found it tricky to use when writing markup and code, but you may become quite adept at it. A little rectangle pops up on your screen, you mouse over it and choose, let’s say, “Double-click.” Until you change that choice, if you mouse over a link or anything else, it will double-click it for you. All you do is glide your mouse around. Awkward for a day or two, but you’ll pick it up quickly. Though you can use it all day for work, even if you just use this for browsing LOLcats or Gary Vaynerchuk’s YouTube videos, it will help you by giving your fingers a sweet break. But here’s the sad news. The developer who invented this died a few years ago. (Yes, I used to speak to him on the phone.) While it is for sale, it isn’t compatible with Mac OS X Lion or anything subsequent. PowerPC strikes again. His site is still up. Demos for use with older software can be downloaded free at his old site, or at The Human Solution. Perhaps an enterprising developer can invent something that would provide this help, without interfering with patents. Rumor has it among ergonomic retailers (yes, I’m like a police dog sniffing my way to a criminal once I head down a trail) that his company was purchased by a company in China, with no update in sight. 4. Use built-in features That little microphone icon that comes up alongside the keyboard on your iPhone allows you to speak your message instead of incessantly thumbing it. I believe it works in any program that uses the keyboard. It’s not Siri. She’s for other things, like having a personal relationship with an inanimate object. Apple even has a good section on ergonomics. You think I’m intense about this subject? To improve your repetitive stress, Apple doesn’t want you to use oral contraceptives, alcohol, or tobacco, to which I say, “Have as much sex, bacon, and chocolate as possible to make up for it.” Apple’s info even has illustrations of things like a faucet dripping into what is labeled a bucket full of “TRAUMA.” Sounds like upgrading to Yosemite, but I digress. 5. Take breaks If it’s a game or other non-essential activity, take a break for a month. Fine, now that I’ve called games non-essential, I suppose you’ll all unfollow me on Twitter. 6. Whether you are sore or not, do stretches throughout the day This is a big one. Really big. The best book on the subject of repetitive stress injuries is Conquering Carpal Tunnel Syndrome and Other Repetitive Strain Injuries: A Self-Care Program by Sharon J. Butler. Don’t worry, most of it is illustrations. Pretend it’s a graphic novel. I’m notorious for never reading instructions, and who on earth reads the introduction of a book, unless they wrote it? I wrote a book a long time ago, and I bet my house, husband, and life savings that my own parents never read the intro. Well, I did read the intro to this book, and you should, too. Stretching correctly, in a way that doesn’t further hurt you, that keeps you flexible if you aren’t injured, that actually heals you, calls for precision. Read and you’ll see. The key is to stretch just until you start to feel the stretch, even if that’s merely a tiny movement. Don’t force anything past that point. Kindly nurse yourself back to health, or nurture your still-healthy body by stretching. Over the following days, weeks, months, you’ll be moving well past that initial stretch point. The book is brimming with examples. You only have to pick a few stretches, if this is too much to handle. Do it every single day. I can tell you some of the best ones for me, but it depends on the person. You’ll also discover in Butler’s book that areas that you think are the problem are sometimes actually adjacent to the muscle or tendon that is the source of the problem. Add a few stretches or two for that area, too. But please follow the instructions in the introduction. If you overdo it, or perform some other crazy-ass hijinks, as I would be tempted to do, I am not responsible for your outcome. I give you fair warning that I am not a healthcare provider. I’m just telling you as a friend, an untrained one, at that, who has been through this experience. 7. Follow good habits Develop habits like drinking lots of water (which helps with lactic acid buildup in muscles), looking away from the computer for twenty seconds every twenty to thirty minutes, eating right, and probably doing everything else your mother told you to do. Maybe this is a good time to bring up flossing your teeth, and going outside to play instead of watching TV. As your mom would say, “It’s a beautiful day outside, what are you kids doing in here?” 8. Speak instead of writing, if you can Amber Simmons, who is very smart and funny, once tweeted in front of the whole world that, “@carywood is a Skype whore.” I was always asking people on Twitter if we could Skype instead of using iChat or exchanging emails. (I prefer the audio version so I don’t have to, you know, do something drastic like comb my hair.) Keyboarding is tough on hands, whether you notice it or not at the time, and when doing rapid-fire back-and-forthing with people, you tend to speed up your typing and not take any breaks. This is a hand-killer. Voice chats have made such a difference for me that I am still a rabid Skype whore. Wait, did I say that out loud? Speak your text or emails, using Dragon Dictate or other software. In about 2005, accessibility and user experience design expert, Derek Featherstone, in Canada, and I, at home, chatted over the internet, each of us using a different voice-to-text program. The programs made so many mistakes communicating with each other that we began that sort of endless, tearful laughing that makes you think someone may need to call an ambulance. This type of software has improved quite a bit over the years, thank goodness. Lack of accessibility of any kind isn’t funny to Derek or me or to anyone who can’t use the web without pain. 9. Watch your position For example, if you lift up your arms to use the computer, or stare down at your laptop, you’ll need to rearrange your equipment. The internet has a lot of information about ideal ergonomic work areas. Please use a keyboard drawer. Be sure to measure the height carefully so that even a tented keyboard, like the one I recommend, will fit. I also recommend getting the version of the Freestyle with palm supports. Just these two measures did much to help both Jen Simmons and me. 10. If you need to take anti-inflammatories, stop working If you are all drugged up on ibuprofen, and pounding and clicking like mad, your body will not know when you are tired or injuring yourself. I don’t recommend taking these while using your computing devices. Perhaps just take it at night, though I’m not a fan of that category of medications. Check with your healthcare provider. At least ibuprofen is an anti-inflammatory, which may help you. In contrast, acetaminophen (paracetamol) only makes your body think it’s not in pain. Ice is great, as is switching back and forth between ice and heat. But again, if you need ice and ibuprofen you really need to take a major break. 11. Don’t forget the rest of your body I’ve zeroed in on my personal area of knowledge and experience, but you may be setting yourself up for problems in other areas of your body. There’s what is known to bad writers as “a veritable cornucopia” of information on the web about how to help the rest of your body. A wee bit of research on the web and you’ll discover simple exercises and stretches for the rest of your potential catastrophic areas: your upper back, your lower back, your legs, ankles, and eyes. Do gentle stretches, three or four times a day, rather than powering your way through. Ease into new equipment such as standing desks. Stretch those newly challenged areas until your body adapts. Pay attention to your body, even though I too often forget mine. 12. Remember the children Kids are using equipment to play highly addictive games or to explore amazing software, and if these call for repetitive motions, children are being set up for future injuries. They’ll grab hold of something, as parents out there know, and play it 3,742 times. That afternoon. Perhaps by the time they are adults, everything will just be holograms and mind-reading, but adult fingers and hands are used for most things in life, not just computing devices and phones with keyboards sized for baby chipmunks. I’ll be watching you Quickly now, while I (possibly) have your attention. Don’t move a muscle. Is your neck tense? Are you unconsciously lifting your shoulders up? How long since you stopped staring at the screen? How bright is your screen? Are you slumping (c’mon now, ‘fess up) and inviting sciatica problems? Do you have to turn your hands at an angle relative to your wrist in order to type? Uh-oh. That’s a bad one. Your hands, wrists, and forearms should be one straight line while keyboarding. Future you is begging you to change your ways. Don’t let your #ThrowbackThursday in 2020 say, “Here’s a photo from when I used to be able to do so many wonderful things that I can’t do now.” And, whatever you do, don’t try for even a nanosecond to push through the pain, or the next thing you know, you’ll be an unpaid extra in The Expendables 7.",2014,Carolyn Wood,carolynwood,2014-12-06T00:00:00+00:00,https://24ways.org/2014/dont-push-through-the-pain/,business 264,Dynamic Social Sharing Images,"Way back when social media was new, you could be pretty sure that whatever you posted would be read by those who follow you. If you’d written a blog post and you wanted to share it with those who follow you, you could post a link and your followers would see it in their streams. Oh heady days! With so many social channels and a proliferation of content and promotions flying past in everyone’s streams, it’s no longer enough to share content on social media, you have to actively sell it if you want it to be seen. You really need to make the most of every opportunity to catch a reader’s attention if you’re trying to get as many eyes as possible on that sweet, sweet social content. One of the best ways to grab attention with your posts or tweets is to include an image. There’s heaps of research that says that having images in your posts helps them stand out to followers. Reports I found showed figures from anything from 35% to 150% improvement from just having image in a post. Unfortunately, the details were surrounded with gross words like engagement and visual marketing assets and so I had to close the page before I started to hate myself too much. So without hard stats to quote, we’ll call it a rule of thumb. The rule of thumb is that posts with images will grab more attention than those without, so it makes sense that when adding pages to a website, you should make sure that they have social media sharing images associated with them. Adding sharing images The process for declaring an image to be used in places like Facebook and Twitter is very simple, and at this point is familiar to many of us. You add a meta tag to the head of the page to point to the location of the image to use. When a link to the page is added to a post, the social network will fetch the page, look for the meta tag and then use the image you specified. There’s a good post on this over at CSS-Tricks if you need to bone up on the details of this and other similar meta tags for social media sharing. This is all fine and well for content that has a very obvious choice of image to go along with it, but what if you don’t necessarily have an image? One approach is to use stock photography, but that’s not going to be right for every situation. This was something we faced with 24 ways in 2017. We wanted to add images to the tweets we post each day announcing a new article. Some articles have images, but not all, and there tended not to be any consistency in terms of imagery from one article to the next. We always have an author photograph, but those don’t usually lend themselves directly to being the main ‘hero’ image for an article. Putting his thinking cap on, Paul came up with a design for an image that used the author photo along with a quote extracted from the article. One of the hand-made sharing images from 2017 Each day we would pick a quote from the article, and Paul would manually compose an image to be uploaded to the site. The results were great, but the whole process was a bit too labour intensive and relied on an individual (Paul) being available each day to do the work. I thought we could probably improve this. Hatching a new plan One initial idea I came up with was to script the image editor to dynamically build a new image by pulling content from our database. Sketch has plugins available to pull JSON content into a design, and our CMS can easily output JSON data, so that was one possibility. The more I thought about this and how much I wish graphic design tools worked just a little bit more like CSS, the obvious solution hit me. We should just build it with CSS! In fact, as the author name and image already exist in our CMS, and the visual styling is based on the design of the website, couldn’t this just be another page on the site generated by the CMS? Breaking it down, I figured the steps needed would be something like: Create the CSS to lay out a component that could be turned into an image Add a new field to articles in the CMS to hold a handpicked quote Build a new article template in the CMS to output the author name and quote dynamically for any article … um … screenshot? I thought I’d get cracking and see if I could figure out the final steps later. Building the page The first thing to tackle was the basic HTML and CSS to lay out the components for our image. That bit was really easy, as I just asked Paul to do it. Everyone should have a Paul. Paul’s code uses a fixed dimension container in the HTML, set to 600 × 315px. This is to make it the correct aspect ratio for Facebook’s recommended image size. It’s useful to remember here that it doesn’t need to be responsive or robust, as the page only needs to lay out correctly for a screenshot and a fixed size in a known browser. With the markup and CSS in place, I turned this into a new template. Our CMS can easily display content through any number of templates, so I created a version of the article template that was totally stripped down. It only included the author details and the quote, along with Paul’s markup. I also added the quote as a new field on the article in the CMS, so each ‘image’ could be quickly and easily customised in the editing process. I added a new field to the article template to capture the quote. With very little effort, we quickly had a page to dynamically generate our ‘image’ right from the CMS. You can see any of them by adding /sharing onto the end of an article URL for any 2018 article. Our automatically generated layout direct from the CMS It soon became clear that the elusive Step 4 was going to be the tricky part. I can create a small page on the site that looks like an image, but how should I go about turning it into one? An obvious route is to screenshot the page by hand, but that’s going back to some of the manual steps I was trying to eliminate, and also opens up a possibility for errors to be made. But it did lead me to the thought… how could I automatically take a screenshot? Enter Puppeteer Puppeteer is a Node.js library that provides a nice API onto Headless Chrome. What is Headless Chrome, you ask? It’s just a version of the Chrome browser than runs from the command line without ever drawing anything to a user interface window. It loads pages, renders CSS, runs JavaScript, pretty much every normal thing that Chrome on the desktop does, but without a clicky user interface. Headless Chrome can be used for all sorts of things such as running automated tests on front-end code after making changes, or… get this… rendering pages that can be used for screenshots. The actual process of writing some code to control Chrome and to take the screenshot is where Puppeteer comes in. Puppeteer puts a friendly layer in front of big old scary Chrome to enable us to interact with it using simple JavaScript code running in Node. Using Puppeteer, I can write a small script that will repeatably turn a URL into an image. So simple is it to do this, that’s it’s actually Puppeteer’s ‘hello world’ example. First you install Puppeteer. It downloads a compatible headless browser (actually Chromium) as a dependancy, so you don’t need to worry about installing that. At the command line: npm i puppeteer Then save a new file as example.js with this code: const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://example.com'); await page.screenshot({path: 'example.png'}); await browser.close(); })(); and then run it using Node: node example.js This will output an image file example.png to disk, which contains a screenshot of, in this case https://example.com. The logic of the code is reasonably easy to follow: Launch a browser Open up a new page Goto a URL Take a screenshot Close the browser The async function and await keywords are a way to have the script pause and wait for normally asynchronous code to return before proceeding. That’s useful with actions like loading a web page that might take some time to complete. They’re used with Promises, and the effect is to make asynchronous code behave as if it’s synchronous. You can read more about async and await at MDN if you’re interested. That’s a good proof-of-concept using the basic Puppeteer example. I can take a screenshot of a URL! But what happens if I put the URL of my new special page in there? Our content is up in the corner of the image with lots of empty space. That’s not great. It’s okay, but not great. It looks like that, by default, Puppeteer takes a screenshot with a resolution of 800 × 600, so we need to find out how to adjust that. Fortunately, the docs aren’t the worst and I was able to find the page.setViewport() method pretty easily. const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://24ways.org/2018/clip-paths-know-no-bounds/sharing'); await page.setViewport({ width: 600, height: 315 }); await page.screenshot({path: 'example.png'}); await browser.close(); })(); This worked! The screenshot is now 600 × 315 as expected. That’s exactly what we asked for. Trouble is, that’s a bit low res and it is nearly 2019 after all. While in those docs, I noticed the deviceScaleFactor option that can be passed to page.setViewport(). Setting that to 2 gives us an image of the same area of the screen, but with twice as many pixels. await page.setViewport({ width: 600, height: 315, deviceScaleFactor: 2 }); Perfect! We now have a programmatic way of turning a URL into an image. Improving the script Rather than having a script with a fixed URL in it that outputs an image called example.png, the next step is to make that a bit more dynamic. The aim here is to have a script that we can run with a URL as an argument and have it output an image for that one page. That way we can run it manually, or hook it into part of our site’s build process to automate the generation of the image. Our goal is to call the script like this: node shoot-sharing-image.js https://24ways.org/2018/clip-paths-know-no-bounds/ And I want the image to come out with the name clip-paths-know-no-bounds.png. To do that, I need to have my script look for command arguments, and then to split the URL up to grab the slug from it. // Get the URL and the slug segment from it const url = process.argv[2]; const segments = url.split('/'); // Get the second-to-last segment (the slug) const slug = segments[segments.length-2]; We can then use these variables later in the script, remembering to add sharing back onto the end of the URL to get our dedicated page. (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto(url + 'sharing'); await page.setViewport({ width: 600, height: 315, deviceScaleFactor: 2 }); await page.screenshot({path: slug + '.png'}); await browser.close(); })(); Once you’re generating the image with Node, there’s all sorts of things you can do with it. An obvious step is to move it to the correct location within your site or project. You can also run optimisations on the file. I’m using imagemin with pngquant to reduce the file size a little. const imagemin = require('imagemin'); const imageminPngquant = require('imagemin-pngquant'); await imagemin([slug + '.png'], 'build', { plugins: [ imageminPngquant({quality: '75-90'}) ] }); You can see the completed example as a gist. Integrating it with your CMS So we now have a command we can run to take a URL and generate a custom image for that URL. It’s in a format that can be called by any sort of build script, or triggered from a publishing hook in a CMS. Exactly how you do that is going to depend on the way your site is built and the technology stack you’re using, but it’s likely not too hard as long as you can run a command as part of the process. For 24 ways this year, I’ve been running the script by hand once each article is ready. My script adds the file to a git repo and pushes to a deployment remote that is configured to automatically deploy static assets to our server. Along with our theme of making incremental improvements, next year I’ll look to automate this one step further. We may also look at having a few slightly different layouts to choose from, so that each day isn’t exactly the same as the last. Interestingly, we could even try some A/B tests to see if there’s any particular format of image or type of quote that does a better job of grabbing attention. There are lots of possibilities! By using a bit of ingenuity, some custom CMS templates, and the very useful Puppeteer project, we’ve been able to reliably produce dynamic social media sharing images for all of our articles. In doing so, we reduced the dependancy on any individual for producing those images, and opened up a world of possibilities in how we use those images. I hope you’ll give it a try!",2018,Drew McLellan,drewmclellan,2018-12-24T00:00:00+00:00,https://24ways.org/2018/dynamic-social-sharing-images/,code 120,Easier Page States for Wireframes,"When designing wireframes for web sites and web apps, it is often overlooked that the same ‘page’ can look wildly different depending on its context. A logged-in page will look different from a logged-out page; an administrator’s view may have different buttons than a regular user’s view; a power user’s profile will be more extensive than a new user’s. These different page states need designing at some point, especially if the wireframes are to form a useful communication medium between designer and developer. Documenting the different permutations can be a time consuming exercise involving either multiple pages in one’s preferred box-and-arrow software, or a fully fledged drawing containing all the possible combinations annotated accordingly. Enter interactive wireframes and Polypage Interactive wireframes built in HTML are a great design and communication tool. They provide a clickable prototype, running in the browser as would the final site. As such they give a great feel for how the site will be to use. Once you add in the possibilities of JavaScript and a library such as jQuery, they become even more flexible and powerful. Polypage is a jQuery plugin which makes it really easy to design multiple page states in HTML wireframes. There’s no JavaScript knowledge required (other than cutting and pasting in a few lines). The page views are created by simply writing all the alternatives into your HTML page and adding special class names to apply state and conditional view logic to the various options. When the page is loaded Polypage automatically detects the page states defined by the class names and creates a control bar enabling the user to toggle page states with the click of a mouse or the clack of a keyboard. Using cookies by way of the jQuery cookie plugin, Polypage retains the view state throughout your prototype. This means you could navigate through your wireframes as if you were logged out; as if you were logged in as an administrator; with notes on or off; or with any other view or state you might require. The possibilities are entirely up to you. How does it work? Firstly you need to link to jQuery, the jQuery cookie plugin and to Polypage. Something like this: Then you need to initialise Polypage on page load using something along these lines: Next you need to define the areas of your wireframe which are particular to a given state or view. Do this by applying classes beginning with pp_. Polypage will ignore all other classes in the document. The pp_ prefix should be followed by a state name. This can be any text string you like, bearing in mind it will appear in the control bar. Typical page states might include ‘logged_in’, ‘administrator’ or ‘group_owner’. A complete class name would therefore look something like pp_logged_in. Examples If a user is logged in, you might want to specify an option for him or her to sign out. Using Polypage, this could be put in the wireframe as follows: Sign out Polypage will identify the pp_logged_in class on the link and hide it (as the ‘Sign out’ link should only be shown when the page is in the ‘logged in’ view). Polypage will then automatically write a ‘logged in’ toggle to the control bar, enabling you to show or hide the ‘Sign out’ link by toggling the ‘logged in’ view. The same will apply to all content marked with a pp_logged_in class. States can also be negated by adding a not keyword to the class name. For example you might want to provide a log in link for users who are not signed in. Using Polypage, you would insert the not keyword after the pp prefix as follows: Login Again Polypage identifies the pp prefix but this time sees that the ‘Login’ link should not be shown when the ‘logged in’ state is selected. States can also be joined together to add some basic logic to pages. The syntax follows natural language and uses the or and and keywords in addition to the afore-mentioned not. Some examples would be pp_logged_in_and_admin, pp_admin_or_group_owner and pp_logged_in_and_not_admin. Finally, you can set default states for a page by passing an array to the polypage.init() function like this: $.polypage.init(['logged_in', 'admin']); You can see a fully fledged example in this fictional social network group page. The example page defaults to a logged in state. You can see the logged out state by toggling ‘logged in’ off in the Polypage control bar. There are also views specified for a group member, a group admin, a new group and notes. Where can I get hold of it? You can download the current version from GitHub. Polypage was originally developed by Clearleft and New Bamboo, with particular contributions from Andy Kent and Natalie Downe. It has been used in numerous real projects, but it is still an early release so there is bound to be room for improvement. We’re pleased to say that Polypage is now an open source project so any feedback, particularly by way of actual improvements, is extremely welcome.",2008,Richard Rutter,richardrutter,2008-12-11T00:00:00+00:00,https://24ways.org/2008/easier-page-states-for-wireframes/,process 101,Easing The Path from Design to Development,"As a web developer, I have the pleasure of working with a lot of different designers. There has been a lot of industry discussion of late about designers and developers, focusing on how different we sometimes are and how the interface between our respective phases of a project (that is to say moving from a design phase into production) can sometimes become a battleground. I don’t believe it has to be a battleground. It’s actually more like being a dance partner – our steps are different, but as long as we know our own part and have a little knowledge of our partner’s steps, it all goes together to form a cohesive dance. Albeit with less spandex and fewer sequins (although that may depend on the project in question). As the process usually flows from design towards development, it’s most important that designers have a little knowledge of how the site is going to be built. At the specialist web development agency I’m part of, we find that designs that have been well considered from a technical perspective help to keep the project on track and on budget. Based on that experience, I’ve put together my checklist of things that designers should consider before handing their work over to a developer to build. Layout One rookie mistake made by traditionally trained designers transferring to the web is to forget a web browser is not a fixed medium. Unlike designing a magazine layout or a piece of packaging, there are lots of available options to consider. Should the layout be fluid and resize with the window, or should it be set to a fixed width? If it’s fluid, which parts expand and which not? If it’s fixed, should it sit in the middle of the window or to one side? If any part of the layout is going to be flexible (get wider and narrower as required), consider how any graphics are affected. Images don’t usually look good if displayed at anything other that their original size, so should they behave? If a column is going to get wider than it’s shown in the Photoshop comp, it may be necessary to provide separate wider versions of any background images. Text size and content volume A related issue is considering how the layout behaves with both different sizes of text and different volumes of content. Whilst text zooming rather than text resizing is becoming more commonplace as the default behaviour in browsers, it’s still a fundamentally important principal of web design that we are suggesting and not dictating how something should look. Designs must allow for a little give and take in the text size, and how this affects the design needs to be taken into consideration. Keep in mind that the same font can display differently in different places and platforms. Something as simple as Times will display wider on a Mac than on Windows. However, the main impact of text resizing is the change in how much vertical space copy takes up. This is particularly important where space is limited by the design (making text bigger causes many more problems than making text smaller). Each element from headings to box-outs to navigation items and buttons needs to be able to expand at least vertically, if not horizontally as well. This may require some thought about how elements on the page may wrap onto new lines, as well as again making sure to provide extended versions of any graphical elements. Similarly, it’s rare theses days to know exactly what content you’re working with when a site is designed. Many, if not most sites are designed as a series of templates for some kind of content management system, and so designs cannot be tweaked around any specific item of content. Designs must be able to cope with both much greater and much lesser volumes of content that might be thrown in at the lorem ipsum phase. Particular things to watch out for are things like headings (how do they wrap onto multiple lines) and any user-generated items like usernames. It can be very easy to forget that whilst you might expect something like a username to be 8-12 characters, if the systems powering your site allow for 255 characters they’ll always be someone who’ll go there. Expect them to do so. Again, if your site is content managed or not, consider the possibility that the structure might be expanded in the future. Consider how additional items might be added to each level of navigation. Whilst it’s rarely desirable to make significant changes without revisiting the site’s information architecture more thoroughly, it’s an inevitable fact of life that the structure needs a little bit of flexibility to change over time. Interactions with and without JavaScript A great number of sites now make good use of JavaScript to streamline the user interface and make everything just that touch more usable. Remember, though, that any developer worth their salt will start by building the interface without JavaScript, get it all working, and then layer that JavaScript on top. This is to allow for users viewing the site without JavaScript available or enabled in their browser. Designers need to consider both states of any feature they’re designing – how it looks and functions with and without JavaScript. If the feature does something fancy with Ajax, consider how the same can be achieved with basic HTML forms, links and intermediary pages. These all need to be designed, because this is how some of your users will interact with the site. Logged in and logged out states When designing any type of web application or site that has a membership system – that is to say users can create an account and log into the site – the design will need to consider how any element is presented in both logged in and logged out states. For some items there’ll be no difference, whereas for others there may be considerable differences. Should an item be hidden completely not logged out users? Should it look different in some way? Perhaps it should look the same, but prompt the user to log in when they interact with it. If so, what form should that prompt take on and how does the user progress through the authentication process to arrive back at the task they were originally trying to complete? Couple logged in and logged out states with the possible absence of JavaScript, and every feature needs to be designed in four different states: Logged out with JavaScript available Logged in with JavaScript available Logged out without JavaScript available Logged in without JavaScript available Fonts There are three main causes of war in this world; religions, politics and fonts. I’ve said publicly before that I believe the responsibility for this falls squarely at the feet of Adobe Photoshop. Photoshop, like a mistress at a brothel, parades a vast array of ropey, yet strangely enticing typefaces past the eyes of weak, lily-livered designers, who can’t help but crumble to their curvy charms. Yet, on the web, we have to be a little more restrained in our choice of typefaces. The purest solution is always to make the best use of the available fonts, but this isn’t always the most desirable solution from a design point of view. There are several technical solutions such as techniques that utilise Flash (like sIFR), dynamically generated images and even canvas in newer browsers. Discuss the best approach with your developer, as every different technique has different trade-offs, and this may impact the design in other ways. Messaging Any site that has interactive elements, from a simple contact form through to fully featured online software application, involves some kind of user messaging. By this I mean the error messages when something goes wrong and the success and thank-you messages when something goes right. These typically appear as the result of an interaction, so are easy to forget and miss off a Photoshop comp. For every form, consider what gets displayed to the user if they make a mistake or miss something out, and also what gets displayed back when the interaction is successful. What do they see and where do the go next? With Ajax interactions, the user doesn’t get any visual feedback of the site waiting for a response from the server unless you design it that way. Consider using a ‘waiting’ or ‘in progress’ spinner to give the user some visual feedback of any background processes. How should these look? How do they animate? Similarly, also consider the big error pages like a 404. With luck, these won’t often be seen, but it’s at the point that they are when careful design matters the most. Form fields Depending on the visual style of your site, the look of a browser’s default form fields and buttons can sometimes jar. It’s understandable that many a designer wants to change the way they look. Depending on the browser in question, various things can be done to style form fields and their buttons (although it’s not as flexible as most would like). A common request is to replace the default buttons with a graphical button. This is usually achievable in most cases, although it’s not easy to get a consistent result across all browsers – particularly when it comes to vertical positioning and the space surrounding the button. If the layout is very precise, or if space is at a premium, it’s always best to try and live with the browser’s default form controls. Whichever way you go, it’s important to remember that in general, each form field should have a label, and each form should have a submit button. If you find that your form breaks either of those rules, you should double check. Practical tips for handing files over There are a couple of basic steps that a design can carry out to make sure that the developer has the best chance of implementing the design exactly as envisioned. If working with Photoshop of Fireworks or similar comping tool, it helps to group and label layers to make it easy for a developer to see which need to be turned on and off to get to isolate parts of the page and different states of the design. Also, if you don’t work in the same office as your developer (and so they can’t quickly check with you), provide a PDF of each page and state so that your developer can see how each page should look aside from any confusion with quick layers are switched on or off. These also act as a handy quick reference that can be used without firing up Photoshop (which can kill both productivity and your machine). Finally, provide a colour reference showing the RGB values of all the key colours used throughout the design. Without this, the developer will end up colour-picking from the comps, and could potentially end up with different colours to those you intended. Remember, for a lot of developers, working in a tool like Photoshop is like presenting a designer with an SSH terminal into a web server. It’s unfamiliar ground and easy to get things wrong. Be the expert of your own domain and help your colleagues out when they’re out of their comfort zone. That goes both ways. In conclusion When asked the question of how to smooth hand-over between design and development, almost everyone who has experienced this situation could come up with their own list. This one is mine, based on some of the more common experiences we have at edgeofmyseat.com. So in bullet point form, here’s my checklist for handing a design over. Is the layout fixed, or fluid? Does each element cope with expanding for larger text and more content? Are all the graphics large enough to cope with an area expanding? Does each interactive element have a state for with and without JavaScript? Does each element have a state for logged in and logged out users? How are any custom fonts being displayed? (and does the developer have the font to use?) Does each interactive element have error and success messages designed? Do all form fields have a label and each form a submit button? Is your Photoshop comp document well organised? Have you provided flat PDFs of each state? Have you provided a colour reference? Are we having fun yet?",2008,Drew McLellan,drewmclellan,2008-12-01T00:00:00+00:00,https://24ways.org/2008/easing-the-path-from-design-to-development/,process 314,Easy Ajax with Prototype,"There’s little more impressive on the web today than a appropriate touch of Ajax. Used well, Ajax brings a web interface much closer to the experience of a desktop app, and can turn a bear of an task into a pleasurable activity. But it’s really hard, right? It involves all the nasty JavaScript that no one ever does often enough to get really good at, and the browser support is patchy, and urgh it’s just so much damn effort. Well, the good news is that – ta-da – it doesn’t have to be a headache. But man does it still look impressive. Here’s how to amaze your friends. Introducing prototype.js Prototype is a JavaScript framework by Sam Stephenson designed to help make developing dynamic web apps a whole lot easier. In basic terms, it’s a JavaScript file which you link into your page that then enables you to do cool stuff. There’s loads of capability built in, a portion of which covers our beloved Ajax. The whole thing is freely distributable under an MIT-style license, so it’s good to go. What a nice man that Mr Stephenson is – friends, let us raise a hearty cup of mulled wine to his good name. Cheers! sluurrrrp. First step is to download the latest Prototype and put it somewhere safe. I suggest underneath the Christmas tree. Cutting to the chase Before I go on and set up an example of how to use this, let’s just get to the crux. Here’s how Prototype enables you to make a simple Ajax call and dump the results back to the page: var url = 'myscript.php'; var pars = 'foo=bar'; var target = 'output-div'; var myAjax = new Ajax.Updater(target, url, {method: 'get', parameters: pars}); This snippet of JavaScript does a GET to myscript.php, with the parameter foo=bar, and when a result is returned, it places it inside the element with the ID output-div on your page. Knocking up a basic example So to get this show on the road, there are three files we need to set up in our site alongside prototype.js. Obviously we need a basic HTML page with prototype.js linked in. This is the page the user interacts with. Secondly, we need our own JavaScript file for the glue between the interface and the stuff Prototype is doing. Lastly, we need the page (a PHP script in my case) that the Ajax is going to make its call too. So, to that basic HTML page for the user to interact with. Here’s one I found whilst out carol singing: Easy Ajax
As you can see, I’ve linked in prototype.js, and also a file called ajax.js, which is where we’ll be putting our glue. (Careful where you leave your glue, kids.) Our basic example is just going to take a name and then echo it back in the form of a seasonal greeting. There’s a form with an input field for a name, and crucially a DIV (greeting) for the result of our call. You’ll also notice that the form has a submit button – this is so that it can function as a regular form when no JavaScript is available. It’s important not to get carried away and forget the basics of accessibility. Meanwhile, back at the server So we need a script at the server which is going to take input from the Ajax call and return some output. This is normally where you’d hook into a database and do whatever transaction you need to before returning a result. To keep this as simple as possible, all this example here will do is take the name the user has given and add it to a greeting message. Not exactly Web 2-point-HoHoHo, but there you have it. Here’s a quick PHP script – greeting.php – that Santa brought me early. Season's Greetings, $the_name!

""; ?> You’ll perhaps want to do something a little more complex within your own projects. Just sayin’. Gluing it all together Inside our ajax.js file, we need to hook this all together. We’re going to take advantage of some of the handy listener routines and such that Prototype also makes available. The first task is to attach a listener to set the scene once the window has loaded. He’s how we attach an onload event to the window object and get it to call a function named init(): Event.observe(window, 'load', init, false); Now we create our init() function to do our evil bidding. Its first job of the day is to hide the submit button for those with JavaScript enabled. After that, it attaches a listener to watch for the user typing in the name field. function init(){ $('greeting-submit').style.display = 'none'; Event.observe('greeting-name', 'keyup', greet, false); } As you can see, this is going to make a call to a function called greet() onkeyup in the greeting-name field. That function looks like this: function greet(){ var url = 'greeting.php'; var pars = 'greeting-name='+escape($F('greeting-name')); var target = 'greeting'; var myAjax = new Ajax.Updater(target, url, {method: 'get', parameters: pars}); } The key points to note here are that any user input needs to be escaped before putting into the parameters so that it’s URL-ready. The target is the ID of the element on the page (a DIV in our case) which will be the recipient of the output from the Ajax call. That’s it No, seriously. That’s everything. Try the example. Amaze your friends with your 1337 Ajax sk1llz.",2005,Drew McLellan,drewmclellan,2005-12-01T00:00:00+00:00,https://24ways.org/2005/easy-ajax-with-prototype/,code 315,Edit-in-Place with Ajax,"Back on day one we looked at using the Prototype library to take all the hard work out of making a simple Ajax call. While that was fun and all, it didn’t go that far towards implementing something really practical. We dipped our toes in, but haven’t learned to swim yet. So here is swimming lesson number one. Anyone who’s used Flickr to publish their photos will be familiar with the edit-in-place system used for quickly amending titles and descriptions on photographs. Hovering over an item turns its background yellow to indicate it is editable. A simple click loads the text into an edit box, right there on the page. Prototype includes all sorts of useful methods to help reproduce something like this for our own projects. As well as the simple Ajax GETs we learned how to do last time, we can also do POSTs (which we’ll need here) and a whole bunch of manipulations to the user interface – all through simple library calls. Here’s what we’re building, so let’s do it. Getting Started There are two major components to this process; the user interface manipulation and the Ajax call itself. Our set-up is much the same as last time (you may wish to read the first article if you’ve not already done so). We have a basic HTML page which links in the prototype.js file and our own editinplace.js. Here’s what Santa dropped down my chimney: Edit-in-Place with Ajax

Edit-in-place

Dashing through the snow on a one horse open sleigh.

So that’s our page. The editable item is going to be the

called desc. The process goes something like this: Highlight the area onMouseOver Clear the highlight onMouseOut If the user clicks, hide the area and replace with a '; var button = ' OR '; new Insertion.After(obj, textarea+button); Event.observe(obj.id+'_save', 'click', function(){saveChanges(obj)}, false); Event.observe(obj.id+'_cancel', 'click', function(){cleanUp(obj)}, false); } The first thing to do is to hide the object. Prototype comes to the rescue with Element.hide() (and of course, Element.show() too). Following that, we build up the textarea and buttons as a string, and then use Insertion.After() to place our new editor underneath the (now hidden) editable object. The last thing to do before we leave the user to edit is it attach listeners to the Save and Cancel buttons to call either the saveChanges() function, or to cleanUp() after a cancel. In the event of a cancel, we can clean up behind ourselves like so: function cleanUp(obj, keepEditable){ Element.remove(obj.id+'_editor'); Element.show(obj); if (!keepEditable) showAsEditable(obj, true); } Saving the Changes This is where all the Ajax fun occurs. Whilst the previous article introduced Ajax.Updater() for simple Ajax calls, in this case we need a little bit more control over what happens once the response is received. For this purpose, Ajax.Request() is perfect. We can use the onSuccess and onFailure parameters to register functions to handle the response. function saveChanges(obj){ var new_content = escape($F(obj.id+'_edit')); obj.innerHTML = ""Saving...""; cleanUp(obj, true); var success = function(t){editComplete(t, obj);} var failure = function(t){editFailed(t, obj);} var url = 'edit.php'; var pars = 'id=' + obj.id + '&content=' + new_content; var myAjax = new Ajax.Request(url, {method:'post', postBody:pars, onSuccess:success, onFailure:failure}); } function editComplete(t, obj){ obj.innerHTML = t.responseText; showAsEditable(obj, true); } function editFailed(t, obj){ obj.innerHTML = 'Sorry, the update failed.'; cleanUp(obj); } As you can see, we first grab in the contents of the textarea into the variable new_content. We then remove the editor, set the content of the original object to “Saving…” to show that an update is occurring, and make the Ajax POST. If the Ajax fails, editFailed() sets the contents of the object to “Sorry, the update failed.” Admittedly, that’s not a very helpful way to handle the error but I have to limit the scope of this article somewhere. It might be a good idea to stow away the original contents of the object (obj.preUpdate = obj.innerHTML) for later retrieval before setting the content to “Saving…”. No one likes a failure – especially a messy one. If the Ajax call is successful, the server-side script returns the edited content, which we then place back inside the object from editComplete, and tidy up. Meanwhile, back at the server The missing piece of the puzzle is the server-side script for committing the changes to your database. Obviously, any solution I provide here is not going to fit your particular application. For the purposes of getting a functional demo going, here’s what I have in PHP. Not exactly rocket science is it? I’m just catching the content item from the POST and echoing it back. For your application to be useful, however, you’ll need to know exactly which record you should be updating. I’m passing in the ID of my

, which is not a fat lot of use. You can modify saveChanges() to post back whatever information your app needs to know in order to process the update. You should also check the user’s credentials to make sure they have permission to edit whatever it is they’re editing. Basically the same rules apply as with any script in your application. Limitations There are a few bits and bobs that in an ideal world I would tidy up. The first is the error handling, as I’ve already mentioned. The second is that from an idealistic standpoint, I’d rather not be using innerHTML. However, the reality is that it’s presently the most efficient way of making large changes to the document. If you’re serving as XML, remember that you’ll need to replace these with proper DOM nodes. It’s also important to note that it’s quite difficult to make something like this universally accessible. Whenever you start updating large chunks of a document based on user interaction, a lot of non-traditional devices don’t cope well. The benefit of this technique, though, is that if JavaScript is unavailable none of the functionality gets implemented at all – it fails silently. It is for this reason that this shouldn’t be used as a complete replacement for a traditional, universally accessible edit form. It’s a great time-saver for those with the ability to use it, but it’s no replacement. See it in action I’ve put together an example page using the inert PHP script above. That is to say, your edits aren’t committed to a database, so the example is reset when the page is reloaded.",2005,Drew McLellan,drewmclellan,2005-12-23T00:00:00+00:00,https://24ways.org/2005/edit-in-place-with-ajax/,code 238,Everything You Wanted To Know About Gradients (And a Few Things You Didn’t),"Hello. I am here to discuss CSS3 gradients. Because, let’s face it, what the web really needed was more gradients. Still, despite their widespread use (or is it overuse?), the smartly applied gradient can be a valuable contributor to a designer’s vocabulary. There’s always been a tension between the inherently two-dimensional nature of our medium, and our desire for more intensity, more depth in our designs. And a gradient can evoke so much: the splay of light across your desk, the slow decrease in volume toward the end of your favorite song, the sunset after a long day. When properly applied, graded colors bring a much needed softness to our work. Of course, that whole ‘proper application’ thing is the tricky bit. But given their place in our toolkit and their prominence online, it really is heartening to see we can create gradients directly with CSS. They’re part of the draft images module, and implemented in two of the major rendering engines. Still, I’ve always found CSS gradients to be one of the more confusing aspects of CSS3. So if you’ll indulge me, let’s take a quick look at how to create CSS gradients—hopefully we can make them seem a bit more accessible, and bring a bit more art into the browser. Gradient theory 101 (I hope that’s not really a thing) Right. So before we dive into the code, let’s cover a few basics. Every gradient, no matter how complex, shares a few common characteristics. Here’s a straightforward one: I spent seconds hours designing this gradient. I hope you like it. At either end of our image, we have a final color value, or color stop: on the left, our stop is white; on the right, black. And more color-rich gradients are no different: (Don’t ever really do this. Please. I beg you.) It’s visually more intricate, sure. But at the heart of it, we have just seven color stops (red, orange, yellow, and so on), making for a fantastic gradient all the way. Now, color stops alone do not a gradient make. Between each is a transition point, the fail-over point between the two stops. Now, the transition point doesn’t need to fall exactly between stops: it can be brought closer to one stop or the other, influencing the overall shape of the gradient. A tale of two syntaxes Armed with our new vocabulary, let’s look at a CSS gradient in the wild. Behold, the simple input button: There’s a simple linear gradient applied vertically across the button, moving from a bright sunflowerish hue (#FAA51A, for you hex nuts in the audience) to a much richer orange (#F47A20). And here’s the CSS that makes it happen: input[type=submit] { background-color: #F47A20; background-image: -moz-linear-gradient( #FAA51A, #F47A20 ); background-image: -webkit-gradient(linear, 0 0, 0 100%, color-stop(0, #FAA51A), color-stop(1, #F47A20) ); } I’ve borrowed David DeSandro’s most excellent formatting suggestions for gradients to make this snippet a bit more legible but, still, the code above might have turned your stomach a bit. And that’s perfectly understandable—heck, it sort of turned mine. But let’s step through the CSS slowly, and see if we can’t make it a little less terrifying. Verbose WebKit is verbose Here’s the syntax for our little gradient on WebKit: background-image: -webkit-gradient(linear, 0 0, 0 100%, color-stop(0, #FAA51A), color-stop(1, #F47A20) ); Woof. Quite a mouthful, no? Well, here’s what we’re looking at: WebKit has a single -webkit-gradient property, which can be used to create either linear or radial gradients. The next two values are the starting and ending positions for our gradient (0 0 and 0 100%, respectively). Linear gradients are simply drawn along the path between those two points, which allows us to change the direction of our gradient simply by altering its start and end points. Afterward, we specify our color stops with the oh-so-aptly named color-stop parameter, which takes the stop’s position on the gradient (0 being the beginning, and 100% or 1 being the end) and the color itself. For a simple two-color gradient like this, -webkit-gradient has a bit of shorthand notation to offer us: background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#FAA51A), to(#FAA51A) ); from(#FAA51A) is equivalent to writing color-stop(0, #FAA51A), and to(#FAA51A) is the same as color-stop(1, #FAA51A) or color-stop(100%, #FAA51A)—in both cases, we’re simply declaring the first and last color stops in our gradient. Terse Gecko is terse WebKit proposed its syntax back in 2008, heavily inspired by the way gradients are drawn in the canvas specification. However, a different, leaner syntax came to the fore, eventually appearing in a draft module specification in CSS3. Naturally, because nothing on the web was meant to be easy, this is the one that Mozilla has implemented. Here’s how we get gradient-y in Gecko: background-image: -moz-linear-gradient( #FAA51A, #F47A20 ); Wait, what? Done already? That’s right. By default, -moz-linear-gradient assumes you’re trying to create a vertical gradient, starting from the top of your element and moving to the bottom. And, if that’s the case, then you simply need to specify your color stops, delimited with a few commas. I know: that was almost… painless. But the W3C/Mozilla syntax also affords us a fair amount of flexibility and control, by introducing features as we need them. We can specify an origin point for our gradient: background-image: -moz-linear-gradient(50% 100%, #FAA51A, #F47A20 ); As well as an angle, to give it a direction: background-image: -moz-linear-gradient(50% 100%, 45deg, #FAA51A, #F47A20 ); And we can specify multiple stops, simply by adding to our comma-delimited list: background-image: -moz-linear-gradient(50% 100%, 45deg, #FAA51A, #FCC, #F47A20 ); By adding a percentage after a given color value, we can determine its position along the gradient path: background-image: -moz-linear-gradient(50% 100%, 45deg, #FAA51A, #FCC 20%, #F47A20 ); So that’s some of the flexibility implicit in the W3C/Mozilla-style syntax. Now, I should note that both syntaxes have their respective fans. I will say that the W3C/Mozilla-style syntax makes much more sense to me, and lines up with how I think about creating gradients. But I can totally understand why some might prefer WebKit’s more verbose approach to the, well, looseness behind the -moz syntax. À chacun son gradient syntax. Still, as the language gets refined by the W3C, I really hope some consensus is reached by the browser vendors. And with Opera signaling that it will support the W3C syntax, I suppose it falls on WebKit to do the same. Reusing color stops for fun and profit But CSS gradients aren’t all simple colors and shapes and whatnot: by getting inventive with individual color stops, you can create some really complex, compelling effects. Tim Van Damme, whose brain, I believe, should be posthumously donated to science, has a particularly clever application of gradients on The Box, a site dedicated to his occasional podcast series. Now, there are a fair number of gradients applied throughout the UI, but it’s the feature image that really catches the eye. You see, there’s nothing that says you can’t reuse color stops. And Tim’s exploited that perfectly. He’s created a linear gradient, angled at forty-five degrees from the top left corner of the photo, starting with a fully transparent white (rgba(255, 255, 255, 0)). At the halfway mark, he’s established another color stop at an only slightly more opaque white (rgba(255, 255, 255, 0.1)), making for that incredibly gradual brightening toward the middle of the photo. But then he has set another color stop immediately on top of it, bringing it back down to rgba(255, 255, 255, 0) again. This creates that fantastically hard edge that diagonally bisects the photo, giving the image that subtle gloss. And his final color stop ends at the same fully transparent white, completing the effect. Hot? I do believe so. Rocking the radials We’ve been looking at linear gradients pretty exclusively. But I’d be remiss if I didn’t at least mention radial gradients as a viable option, including a modest one as a link accent on a navigation bar: And here’s the relevant CSS: background: -moz-radial-gradient(50% 100%, farthest-side, rgb(204, 255, 255) 1%, rgb(85, 85, 85) 15%, rgba(85, 85, 85, 0) ); background: -webkit-gradient(radial, 50% 100%, 0, 50% 100%, 15, from(rgb(204, 255, 255)), to(rgba(85, 85, 85, 0)) ); Now, the syntax builds on what we’ve already learned about linear gradients, so much of it might be familiar to you, picking out color stops and transition points, as well as the two syntaxes’ reliance on either a separate property (-moz-radial-gradient) or parameter (-webkit-gradient(radial, …)) to shift into circular mode. Mozilla introduces another stand-alone property (-moz-radial-gradient), and accepts a starting point (50% 100%) from which the circle radiates. There’s also a size constant defined (farthest-side), which determines the reach and shape of our gradient. WebKit is again the more verbose of the two syntaxes, requiring both starting and ending points (50% 100% in both cases). Each also accepts a radius in pixels, allowing you to control the skew and breadth of the circle. Again, this is a fairly modest little radial gradient. Time and article length (and, let’s be honest, your author’s completely inadequate grasp of geometry) prevent me from covering radial gradients in much more detail, because they are incredibly powerful. For those interested in learning more, I can’t recommend the references at Mozilla and Apple strongly enough. Leave no browser behind But no matter the kind of gradients you’re working with, there is a large swathe of browsers that simply don’t support gradients. Thankfully, it’s fairly easy to declare a sensible fallback—it just depends on the kind of fallback you’d like. Essentially, gradient-blind browsers will disregard any properties containing references to either -moz-linear-gradient, -moz-radial-gradient, or -webkit-gradient, so you simply need to keep your fallback isolated from those properties. For example: if you’d like to fall back to a flat color, simply declare a separate background-color: .nav { background-color: #000; background-image: -moz-linear-gradient(rgba(0, 0, 0, 0), rgba(255, 255, 255, 0.45)); background-image: -webkit-gradient(linear, 0 0, 0 100%, from(rgba(0, 0, 0, 0)), to(rgba(255, 255, 255, 0.45))); } Or perhaps just create three separate background properties. .nav { background: #000; background: #000 -moz-linear-gradient(rgba(0, 0, 0, 0), rgba(255, 255, 255, 0.45)); background: #000 -webkit-gradient(linear, 0 0, 0 100%, from(rgba(0, 0, 0, 0)), to(rgba(255, 255, 255, 0.45))); } We can even build on this to fall back to a non-gradient image: .nav { background: #000 url(""faux-gradient-lol.png"") repeat-x; background: #000 -moz-linear-gradient(rgba(0, 0, 0, 0), rgba(255, 255, 255, 0.45)); background: #000 -webkit-gradient(linear, 0 0, 0 100%, from(rgba(0, 0, 0, 0)), to(rgba(255, 255, 255, 0.45))); } No matter the approach you feel most appropriate to your design, it’s really just a matter of keeping your fallback design quarantined from its CSS3-ified siblings. (If you’re feeling especially masochistic, there’s even a way to get simple linear gradients working in IE via Microsoft’s proprietary filters. Of course, those come with considerable performance penalties that even Microsoft is quick to point out, so I’d recommend avoiding those. And don’t tell Andy Clarke I told you, or he’ll probably unload his Derringer at me. Or something.) Go forth and, um, gradientify! It’s entirely possible your head’s spinning. Heck, mine is, but that might be the effects of the ’nog. But maybe you’re wondering why you should care about CSS gradients. After all, images are here right now, and work just fine. Well, there are some quick benefits that spring to mind: fewer HTTP requests are needed; CSS3 gradients are easily made scalable, making them ideal for variable widths and heights; and finally, they’re easily modifiable by tweaking a few CSS properties. Because, let’s face it, less time spent yelling at Photoshop is a very, very good thing. Of course, CSS-generated gradients are not without their drawbacks. The syntax can be confusing, and it’s still under development at the W3C. As we’ve seen, browser support is still very much in flux. And it’s possible that gradients themselves have some real performance drawbacks—so test thoroughly, and gradient carefully. But still, as syntaxes converge, and support improves, I think generated gradients can make a compelling tool in our collective belts. The tasteful design is, of course, entirely up to you. So have fun, and get gradientin’.",2010,Ethan Marcotte,ethanmarcotte,2010-12-22T00:00:00+00:00,https://24ways.org/2010/everything-you-wanted-to-know-about-gradients/,code 287,Extracting the Content,"As we throw away our canvas in approaches and yearn for a content-out process, there remains a pain point: the Content. It is spoken of in the hushed tones usually reserved for Lord Voldemort. The-thing-that-someone-else-is-responsible-for-that-must-not-be-named. Designers and developers have been burned before by not knowing what the Content is, how long it is, what style it is and when the hell it’s actually going to be delivered, in internet eons past. Warily, they ask clients for it. But clients don’t know what to make, or what is good, because no one taught them this in business school. Designers struggle to describe what they need and when, so the conversation gets put off until it’s almost too late, and then everyone is relieved that they can take the cop-out of putting up a blog and maybe some product descriptions from the brochure. The Content in content out. I’m guessing, as a smart, sophisticated, and, may I say, nicely-scented reader of the honourable and venerable tradition of 24 ways, that you sense something better is out there. Bunches of boxes to fill in just don’t cut it any more in a responsive web design world. The first question is, how are you going to design something to ensure users have the easiest access to the best Content, if you haven’t defined at the beginning what that Content is? Of course, it’s more than possible that your clients have done lots of user research before approaching you to start this project, and have a plethora of finely tuned Content for you to design with. Have you finished laughing yet? Alright then. Let’s just assume that, for whatever reason of gross oversight, this hasn’t happened. What next? Bringing up Content for the first time with a client is like discussing contraception when you’re in a new relationship. It might be awkward and either party would probably rather be doing something else, but it needs to be broached before any action happens (that, and it’s disastrous to assume the other party has the matter in hand). If we can’t talk about it, how can we expect people to be doing it right and not making stupid mistakes? That being the case, how do we talk about Content? Let’s start by finding a way to talk about it without blushing and scuffing our shoes. And there’s a reason I’ve been treating Content as a Proper Noun. The first step, and I mean really-first-step-way-back-at-the-beginning-of-the-project-while-you-are-still-scoping-out-what-the-hell-you-might-do-for-each-other-and-it’s-still-all-a-bit-awkward-like-a-first-date, is for you to explain to the client how important it is that you, together, work out what is important to your users as part of the user experience design, so that your users get the best user experience. The trouble is that, in most cases, this would lead to blank stares, possibly followed by a light cough and a query about using Comic Sans because it seems friendly. Let’s start by ensuring your clients understand the task ahead. You see, all the time we talk about the Content we do our clients a big disservice. Content is poorly defined. It looms over a project completion point like an unscalable (in the sense of a dozen stacked Kilimanjaros), seething, massive, singular entity. The Content. Defining the problem. We should really be thinking of the Content as ‘contents’; as many parts that come together to form a mighty experience, like hit 90s kids’ TV show Mighty Morphin Power Rangers*. *For those of you who might have missed the Power Rangers, they were five teenagers with attitude, each given crazy mad individual skillz and a coloured lycra suit from an alien overlord. In return, they had to fight a new monster of the week using their abilities and weaponry in sync (even if the audio was not) and then, finally, in thrilling combination as a Humongous Mechanoid Machine of Awesome. They literally joined their individual selves, accessories and vehicles into a big robot. It was a toy manufacturer’s wet dream. So, why do I say Content is like the Power Rangers? Because Content is not just a humongous mecha. It is a combination of well-crafted pieces of contents that come together to form a well-crafted humongous mecha. Of Content. The Red Power Ranger was always the leader. You can imagine your text contents, found on about pages, product descriptions, blog articles, and so on, as being your Red Power Ranger. Maybe your pictures are your Yellow Power Ranger; video is Blue (not used as much as the others, but really impressive when given a good storyline); maybe Pink is your infographics (it’s wrong to find it sexier than the other equally important Rangers, but you kind of do anyway). And so on. These bits of content – Red Text Ranger, Yellow Picture Ranger and others – often join together on a page, like they are teaming up to fight the bad guy in an action scene, and when they all come together (your standard workaday huge mecha) in a launched site, that’s when Content becomes an entity. While you might have a vision for the whole site, Content rarely works that way. Of course, you keep your eye on the bigger prize, the completion of your mega robot, but to get there you need to assemble your working parts, the cogs and springs of contents that will mesh together to finally create your Humongous Mecha of Content. You create parts and join them to form a whole. (It’s rarely seamless; often we need to adjust as we go, but we can create our Mecha’s blueprint by making sure we have all the requisite parts.) The point here is the order these parts were created. No alien overlord plans a Humongous Mechanoid and then thinks, “Gee, how can I split this into smaller fighting units powered by teenagers in snazzy shiny suits?” No toy manufacturer goes into production of a mega robot, made up of model mecha vehicles with detachable arsenal, without thinking how they will easily fit back together to form the ‘Buy all five now to create the mega robot’ set. No good contents are created as a singular entity and chunked up to be slotted in to place any which way, into the body of a site. Think contents, not the Content. Think of contents as smaller units, or as a plural. The Content is what you have at the end. The contents are what you are creating and they are easy to break down. You are no longer scaling the unscalable. You can draw the map and plot the path, page by page, section by section. The page table is your friend To do this, I use a page table. A page table is a simple table template you can create in the word processor of your choice, that you use to tell you everything about the contents of a page – everything except the contents itself. Here’s a page table I created for an employee’s guide to redundancy in the alpha.gov.uk website: Guide to redundancy for employees Page objective: Provide specific information for employees who are facing redundancy about the process, their options and next steps. Source content: directgov page on Redundancy. Scope: In scope Page title An employee’s guide to redundancy Priority content Message: You have rights as an employee facing redundancy Method: A guide written in plain English, with links to appropriate additional content. A video guide (out of scope). Covers the stages of redundancy and rights for those in trade unions and not in trade unions. Glossary of unfamiliar terms. Call to action: Read full guide, act to explore redundancy actions, benefits or new employment. Assets: link to redundancy calculator. Secondary Related items, or popular additional links. Additional tools, such as search and suggestions. location set v not set states microcopy encouraging location set where location may make a difference to the content – ie, Scotland/Northern Ireland. Tertiary Footer and standard links. Content creation: Content exists but was created within the constraints of the previous CMS. Review, correct and edit where necessary. Maintenance: should be flagged for review upon advice from Department of Work and Pensions, and annually. Technology/Publishing/Policy implications: Should be reviewed once the glossary styles have been decided. No video guide in scope at this time, so languages should be simple and screen reader friendly. Reliance on third parties: None, all content and source exists in house. Outstanding questions: None. Download a copy of this page table This particular page table template owes a lot to Brain Traffic’s version found in Kristina Halvorson’s book Content Strategy for the Web. With smaller clients than, say, the government, I might use something a bit more casual. With clients who like timescales and deadlines, I might turn it into a covering sheet, with signatures and agreements from two departments who have to work together to get the piece done on time. I use page tables, and the process of working through them, to reassure clients that I understand the task they face and that I can help them break it down section by section, page stack to page, down to product descriptions and interaction copy. About 80% of my clients break into relieved smiles. Most clients want to work with you to produce something good, they just don’t understand how, and they want you to show them the mountain path on the map. With page tables, clients can understand that with baby steps they can break down their content requirements and commission content they need in time for the designers to work with it (as opposed to around it). If I was Santa, these clients would be on my nice list for sure. My own special brand of Voldemort-content-evilness comes in how I wield my page tables for the other 20%. Page tables are not always thrilling, I’ll admit. Sometimes they get ignored in favour of other things, yet they are crucial to the continual growth and maintenance of a truly content-led site. For these naughty list clients who, even when given the gift of the page table, continually say “Ooh, yes. Content. Right”, I have a special gift. I have a stack of recycled paper under my desk and a cheap black and white laser printer. And I print a blank page table for every conceivable page I can find on the planned redesign. If I’m feeling extra nice, I hole punch them and put them in a fat binder. There is nothing like saying, “This is all the contents you need to have in hand for launch”, and the satisfying thud the binder makes as it hits the table top, to galvanize even the naughtiest clients to start working with you to create the content you need to really create in a content-out way.",2011,Relly Annett-Baker,rellyannettbaker,2011-12-15T00:00:00+00:00,https://24ways.org/2011/extracting-the-content/,content 236,Extreme Design,"Recently, I set out with twelve other designers and developers for a 19th century fortress on the Channel Island of Alderney. We were going to /dev/fort, a sort of band camp for geeks. Our cohort’s mission: to think up, build and finish something – without readily available internet access. Alderney runway, photo by Chris Govias Wait, no internet? Well, pretty much. As the creators of /dev/fort James Aylett and Mark Norman Francis put it: “Imagine a place with no distractions – no IM, no Twitter”. But also no way to quickly look up a design pattern, code sample or source material. Like packing for camping, /dev/fort means bringing everything you’ll need on your back or your hard drive: from long johns to your favourite icon set. We got to work the first night discussing ideas for what we wanted to build. By the time breakfast was cleared up the next morning, we’d settled on Russ’s idea to make the Apollo 13 (PDF) transcript accessible. Days two and three were spent collaboratively planning (KJ style) what features we wanted to build, and unravelling the larger UX challenges of the project. The next five days were spent building it. Within 36 hours of touchdown at Southampton Airport, we launched our creation: spacelog.org The weather was cold, the coal fire less than ideal, food and supplies a hike away, and the process lightning-fast. A week of designing under extreme circumstances called for an extreme process. Some of this was driven by James’s and Norm’s experience running these things, but a lot of it materialised while we were there – especially for our three-strong design team (myself, Gavin O’ Carroll and Chris Govias) who, though we knew each other, had never worked together as a group in this kind of scenario before. The outcome was a pretty spectacular process, with a some key takeaways useful for any small group trying to build something quickly. What it’s like inside the fort /dev/fort has the pressure and pace of a hack day without being a hack day – primarily, no workshops or interruptions‚ but also a different mentality. While hack days are typically developer-driven with a ‘hack first, design later (if at all)’ attitude, James was quick to tell the team to hold off from writing any code until we had a plan. This put a healthy pressure on the design and product folks to slash through the UX problems before we started building. While the fort had definitely more of a hack day feel, all of us were familiar with Agile methods, so we borrowed a few useful techniques such as morning stand-ups and an emphasis on teamwork. We cut some really good features to make our launch date, and chunked the work based on user goals, iterating as we went. What made this design process work? A golden ratio of teams My personal experience both professionally and in free-form situations like this, is a tendency to get/hire a designer. Leaders of businesses, founders of start-ups, organisers of events: one designer is not enough! Finding one ace-blooded designer who can ‘do everything’ will always result in bottleneck and burnout. Like the nuances between different development languages, design is a multifaceted discipline, and very few can claim to be equally strong in every aspect. Overlap in skill set will result in a stronger, more robust interface. More importantly, however, having lots of designers to go around meant that we all had the opportunity to pair with developers, polishing the details that don’t usually get polished. As soon as we launched, the public reception of the design and UX was overwhelmingly positive (proof!). But also, a lot of people asked us who the designer was, attributing it to one person. While it’s important to note that everyone in our team was multitalented (and could easily shift between roles, helping us all stay unblocked), the golden ratio James and Norm devised was two product/developer folks, three interaction designers and eight developers. photo by Ben Firshman Equality inside the fortress walls Something magical about the fort is how everyone leaves the outside world on the drawbridge. Job titles, professional status, Twitter followers, and so on. Like scout camp, a mutual respect and trust is expected of all the participants. Like extreme programming, extreme design requires us all to be equal partners in a collaborative team. I think this is especially worth noting for designers; our past is filled with the clear hierarchy of the traditional studio system which, however important for taste and style, seems less compatible with modern web/software development methods. Being equal doesn’t mean being the same, however. We established clear roles and teams for ourselves on the second day, deferring to that person when a decision needed to be made. As the interface coalesced, the designers and developers took ownership over certain parts to ensure the details got looked after, while staying open to ideas and revisions from the rest of the cohort. Create a space where everyone who enters is equal, but be sure to establish clear roles. Even if it’s just for a short while, the environment will be beneficial. photo by Ben Firshman Hang your heraldry from the rafters Forts and castles are full of lore: coats of arms; paintings of battles; suits of armour. It’s impossible not to be surrounded by these stories, words and ways of thinking. Like the whiteboards on the walls, putting organisational lore in your physical surroundings makes it impossible not to see. Ryan Alexander brought some of those static-cling whiteboard sheets which were quickly filled with use cases; IA; team roles; and, most importantly, a glossary. As soon as we started working on the project, we realised we needed to get clear on what certain words meant: what was a logline, a range, a phase, a key moment? Were the back-end people using these words in the same way design and product was? Quickly writing up a glossary of terms meant everyone was instantly speaking the same language. There was no “Ah, I misunderstood because in the data structure x means y” or, even worse, accidental seepage of technical language into the user interface copy. Put a glossary of your internal terminology somewhere big and fat on the wall. Stand around it and argue until you agree on what it says. Leave it up; don’t underestimate the power of ambient communication and physical reference. Plan more, download less While internet is forbidden inside the fort, we did go on downloading expeditions: NASA photography; code documentation; and so on. The project wouldn’t have been possible without a few trips to the web. We had two lists on the wall: groceries and supplies; internets – “loo roll; Tom Stafford photo“. This changed our usual design process, forcing us to plan carefully and think of what we needed ahead of time. Getting to the internet was a thirty-minute hike up a snow covered cliff to the town airport, so you really had to need it, too. The path to the internet For the visual design, especially, this resulted in more focus up front, and communication between the designers on what assets we required. It made us make decisions earlier and stick with them, creating less distraction and churn later in the process. Try it at home: unplug once you’ve got the things you need. As an artist, it’s easier to let your inner voice shine through if you’re not looking at other people’s work while creating. Social design Finally, our design team experimented with a collaborative approach to wireframing. Once we had collectively nailed down use cases, IA, user journeys and other critical artefacts, we tried a pairing approach. One person drew in Illustrator in real time as the other two articulated what to draw. (This would work equally well with two people, but with three it meant that one of us could jump up and consult the lore on the walls or clarify a technical detail.) The result: we ended up considering more alternatives and quickly rallying around one solution, and resolved difficult problems more quickly. At a certain stage we discovered it was more efficient for one person to take over – this happened around the time when the basic wireframes existed in Illustrator and we’d collectively run through the use cases, making sure that everything was accounted for in a broad sense. At this point, take a break, go have a beer, and give yourself a pat on the back. Put the files somewhere accessible so everyone can use them as their base, and divide up the more detailed UI problems, screens or journeys. At this level of detail it’s better to have your personal headspace. Gavin called this ‘social design’. Chatting and drawing in real time turned what was normally a rather solitary act into a very social process, with some really promising results. I’d tried something like this before with product or developer folks, and it can work – but there’s something really beautiful about switching places and everyone involved being equally quick at drawing. That’s not something you get with non-designers, and frequent swapping of the ‘driver’ and ‘observer’ roles is a key aspect to pairing. Tackle the forest collectively and the trees individually – it will make your framework more robust and your details more polished. Win/win. The return home Grateful to see a 3G signal on our phones again, our flight off the island was delayed, allowing for a flurry of domain name look-ups, Twitter catch-up, and e-mails to loved ones. A week in an isolated fort really made me appreciate continuous connectivity, but also just how unique some of these processes might be. You just never know what crazy place you might be designing from next.",2010,Hannah Donovan,hannahdonovan,2010-12-09T00:00:00+00:00,https://24ways.org/2010/extreme-design/,process 310,Fairytale of new Promise,"There are only four good Christmas songs. I know, yeah, JavaScript or whatever. We’ll get to that in a minute, I promise. First—and I cannot stress this enough— there are four good Christmas songs. You’re free to disagree with me here, of course, but please try to understand that you will be wrong. They don’t all have the most safe-for-work titles; I can’t list all of them here, but if you choose to let your fingers do the walkin’ to your nearest search engine, I will say that one was released by the band FEAR way back in 1982 and one was on Run the Jewels’ self-titled debut album. The lyrics are a hell of a lot worse than the titles, so maybe wait until you get home from work before you queue them up. Wear headphones, if you’ve got thin walls. For my money, though, the two I can reference by name are the top of that small heap: Tom Waits’ Christmas Card from a Hooker in Minneapolis, and The Pogues’ Fairytale of New York. The former once held the honor of being the only good Christmas song—about which which I was also unequivocally correct, right up until I changed my mind. It’s not the song up for discussion today, but feel free to familiarize yourself just the same—I’ll wait. Fairytale of New York—the top of the list—starts out by hinting at some pretty standard holiday fare; dreams and cheer and whatnot. Typical seasonal stuff, so long as you ignore that the story seems to be recounted as a drunken flashback in a jail cell. You can probably make a few guesses at the underlying spirit of the song based on that framing: following a lucky break, our bright-eyed protagonists move to New York in search of fame and fortune, only to quickly descend into bad decisions, name-calling, and vaguely festive chaos. This song speaks to me on a couple of levels, not the least of which is as a retelling of my day-to-day interactions with JavaScript. Each day’s melody might vary a little bit, granted, but the lyrics almost always follow a pretty clear arc toward “PARENTAL ADVISORY: EXPLICIT CONTENT.” You might have heard a similar tune yourself; it goes a little somethin’ like setTimeout(function() { console.log( ""this should be happening last"" ); }, 1000); . Callbacks are calling callbacks calling callbacks and something is happening somewhere, as the JavaScript interpreter plods through our code start-to-finish, line-by-line, step-by-step. If we need to take actions based on the results of something that could take its sweet time resolving, well, we’d better fiddle with the order of things to make sure those actions don’t happen too soon. “But I can see a better time,” as the song says, “when all our dreams come true.” So, with that Pogues brand of holiday spirit squarely in mind—by which I mean that your humble narrator is almost certainly drunk, and may be incarcerated at the time of publication—gather ’round for a story of hope, of hardships, of semi-asynchronous JavaScript programming, and ultimately: of Promise unfulfilled. The Main Thread JavaScript is single-minded, in a manner of speaking. Anything we tell the JavaScript runtime to do goes into a single-file queue; you’ll see it referred to as the “main thread,” or “UI thread.” That thread can be shared by a number of critical browser processes, like rendering and re-rendering parts of the page, and user interactions ranging from the simple—say, highlighting text—to the more complex—interacting with form elements. If that sounds a little scary to you, well, that’s because it is. The more complex our scripts, the more we’re cramming into that single-file main thread, to be processed along with—say—some of our CSS animations. Too much JavaScript clogging up the main thread means a lot of user-facing performance jankiness. Getting away from that single thread is a big part of all the excitement around Web Workers, which allow us to offload entire scripts into their own dedicated background threads—though not without limitations of their own. Outside of Web Workers, that everything-thread is the only game in town: scripts executed one thing at a time, functions calling functions calling functions, taking numbers and crowding up the same deli counter as a user’s interactions—which, in this already strained metaphor, would be ham, I guess? Asynchronous JavaScript Now, those queued actions may include asynchronous things. For example: AJAX callbacks, setTimeout/setInterval, and addEventListener won’t block the main thread while we’re waiting for a request to come back, a timer to tick away, or an event to trigger. Once those things do kick in, though, the actions they’re meant to perform will get shuffled right back into that single-thread queue. There are a couple of places you might have written asynchronously-fired JavaScript, even if you’re not super familiar with the overarching concept: XMLHttpRequest—“AJAX,” if ya nasty—or just kicking off a function once a user triggers a click or mouseenter event. Event-driven development is writ a little larger, with the overall flow of the script dictated by events, both internal and external. Writing event-driven JavaScript applications is a step in the right direction for sure—it won’t cure what ails the main thread, but it does work with the medium in a reasonable way. Event-driven development allows us to manage our use of the main thread in a way that makes sense. If any of this rings a bell for you, the motivation for Promises should feel familiar. For example, a custom init event might kick things off, and fire a create event that applies our classes and restructures our markup which, on completion, fires a bindEvents event to handle all the event listeners for user interaction. There might not sound like much difference between that and one big function that kicks off, manipulates the DOM, and binds our events line-by-line—but in a script of sufficient size and complexity we’re not only provided with a decoupled flow through the script, but obvious touchpoints for future updates and a predictable structure for ongoing maintenance. This pattern falls apart a little where we were still creating, binding, and listening for events in the same top-to-bottom, one-item-at-a-time way—we had to set a listener on a given object before the event fires, or nothing would happen: // Create the event: var event = document.createEvent( ""Event"" ); // Name the event: event.initEvent( ""doTheStuff"", true, true ); // Listen for the custom `doTheStuff` event on `window`: window.addEventListener( ""doTheStuff"", initializeEverything ); // Fire the custom event window.dispatchEvent( event ); This example is a little contrived, and this stuff is a lot more manageable for sure with the addition of a framework, but that’s the basic gist: create and name the event, add a listener for the event, and—after setting our listener—dispatch the event. Events and callbacks aren’t the only game in town for weaving our way in and out of the main thread, though—at least, not anymore. Promises A Promise is, at the risk of sounding sentimental, pure potential—an empty container into which a value eventually results. A Promise can exist in several states: “pending,” while the computation they contain is being performed or “resolved” once that computation is complete. Once resolved, a Promise is “fulfilled” if it gave us back something we expect, or “rejected” if it didn’t. The Promise constructor accepts a callback with two arguments: resolve and reject. We perform an action—asynchronous or otherwise—within that callback. If everything in there has gone according to plan, we call resolve. If something has gone awry, we call reject—with an error, conventionally. To illustrate, let’s tack something together with a pretty decent chance of doing what we don’t want: a promise meant only to give us the number 1, but has a chance of giving us back a 2. No reasonable person would ever do this, of course, but I wouldn’t necessarily put it past me. var promisedOne = new Promise( function( resolve, reject ) { var coinToss = Math.floor( Math.random() * 2 ) + 1; if( coinToss === 1 ) { resolve( coinToss ); } else { reject( new Error( ""That ain’t a one."" ) ); } }); There’s nothing too surprising in there, after you boil it all down. It’s a little return-y, with the exception that we’re flagging results as “as expected” or “something went wrong.” Tapping into that Promise uses another new keyword: then—and as someone who attempts to make sense of JavaScript by breaking it down to plain ol’ human-language, I’m a big fan of this syntax. then is tacked onto our Promise identifier, and does just what it says on the tin: once the Promise is resolved, then do one of two things, both supplied as callbacks: the first in the case of a fulfilled promise, and the second in the case of a rejected one. Those two callbacks will have, as arguments, the results we specified with resolve orreject, respectively. It sounds like a lot in prose, but in code it’s a pretty simple pattern: promisedOne.then( function( result ) { console.log( result ); }, function( error ) { console.error( error ); }); If you’ve spent any time working with AJAX—jQuery-wise, in particular—you’ve seen something like this pattern before: a success callback and an error callback. The state of a promise, once fulfilled or rejected, cannot be changed—any reference we make to promisedOne will have a single, fixed result. It may not look like too much the way I’m using it here, but it’s powerful stuff—a pattern for asynchronously resolving anything. I’ve recently used Promises alongside a script that emulates Font Load Events, to apply webfonts asynchronously and avoid a potential performance hit. Font Face Observer allows us to, as the name implies, determine when the files referenced by our @font-face rules have finished loading. var fontObserver = new FontFaceObserver( ""Fancy Font"" ); fontObserver.check().then(function() { document.documentElement.className += "" fonts-loaded""; }, function( error ) { console.error( error ); }); fontObserver.check() gives us back a Promise, allowing us to chain on a then containing our callbacks for success and failure. We use the fulfilled callback to bolt a class onto the page once the font file has been fully transferred. We don’t bother including an argument in the first function, since we don’t care about the result itself so much as we care that the promise resolved without error—we’re not doing anything with the resolved value, just adding a class to the page. We do include the error argument, since we’ll want to know what happened should something go wrong. Now, this isn’t the tidiest syntax around—at least to my eyes—with those two functions just kinda floating in a then. Luckily there’s an similar alternative syntax; one that I find a bit easier to parse at-a-glance: fontObserver.check() .then(function() { document.documentElement.className += "" fonts-loaded""; }) .catch(function( error ) { console.log( error ); }); The first callback inside then provides us with our success state, while the catch provides us with a single, explicit “something went wrong” callback. The two syntaxes aren’t completely identical in all situations, but for a simple case like this, I find it a little neater. The Common Thread I guess I still owe you an explanation, huh. Not about the JavaScript-whatever; I think I’ve explained that plenty. No, I mean Fairytale of New York, and why it’s perched up there at the top of the four (4) song heap. Fairytale is a sad song, ostensibly. If you follow the main thread—start to finish, line-by-line, step by step— Fairytale is a sad song. And I can see you out there, visions of Die Hard dancing in your heads: “but is it a Christmas song?” Well, for my money, nothing says “holidays” quite like unreliable narration. Shane MacGowan, the song’s author, has placed the first verse about “Christmas Eve in the drunk tank” as happening right after the “lucky one, came in eighteen-to-one”—not at the chronological end of the story. That means the song might not be mostly drunken flashback, but all of it a single, overarching flashback including a Christmas Eve in protective custody. It could be that the man and woman are, together, recounting times long past—good times and bad times—maybe not even in chronological order. Hell, the “NYPD Choir” mentioned in the chorus? There’s no such thing. We’re not big Christmas folks, my family and I. But just the same, every year, the handful of us get together, and every year—like clockwork—there’s a lull in conversation, there’s a sharp exhale, and Ma says “we all made it.” Not to a house, not to a dinner, but through another year, to another Christmas. At this point, without fail, someone starts telling a story—and one begets another, and so on. Sometimes the stories are happy, sometimes they’re sad, more often than not they’re both. Some are about things we were lucky to walk away from, some are about a time when another one of us didn’t. Start-to-finish, line-by-line, step-by-step, the main thread through the year doesn’t change, and maybe there isn’t a whole lot we can do to change it. But by carefully weaving our way in and out of that thread—stories all out of sync and resolving one way or the other, with the results determined by questionably reliable narrators—we can change the way we interact with it and, little by little, we can start making sense of it.",2016,Mat Marquis,matmarquis,2016-12-19T00:00:00+00:00,https://24ways.org/2016/fairytale-of-new-promise/,code 249,Fast Autocomplete Search for Your Website,"Every website deserves a great search engine - but building a search engine can be a lot of work, and hosting it can quickly get expensive. I’m going to build a search engine for 24 ways that’s fast enough to support autocomplete (a.k.a. typeahead) search queries and can be hosted for free. I’ll be using wget, Python, SQLite, Jupyter, sqlite-utils and my open source Datasette tool to build the API backend, and a few dozen lines of modern vanilla JavaScript to build the interface. Try it out here, then read on to see how I built it. First step: crawling the data The first step in building a search engine is to grab a copy of the data that you plan to make searchable. There are plenty of potential ways to do this: you might be able to pull it directly from a database, or extract it using an API. If you don’t have access to the raw data, you can imitate Google and write a crawler to extract the data that you need. I’m going to do exactly that against 24 ways: I’ll build a simple crawler using wget, a command-line tool that features a powerful “recursive” mode that’s ideal for scraping websites. We’ll start at the https://24ways.org/archives/ page, which links to an archived index for every year that 24 ways has been running. Then we’ll tell wget to recursively crawl the website, using the --recursive flag. We don’t want to fetch every single page on the site - we’re only interested in the actual articles. Luckily, 24 ways has nicely designed URLs, so we can tell wget that we only care about pages that start with one of the years it has been running, using the -I argument like this: -I /2005,/2006,/2007,/2008,/2009,/2010,/2011,/2012,/2013,/2014,/2015,/2016,/2017 We want to be polite, so let’s wait for 2 seconds between each request rather than hammering the site as fast as we can: --wait 2 The first time I ran this, I accidentally downloaded the comments pages as well. We don’t want those, so let’s exclude them from the crawl using -X ""/*/*/comments"". Finally, it’s useful to be able to run the command multiple times without downloading pages that we have already fetched. We can use the --no-clobber option for this. Tie all of those options together and we get this command: wget --recursive --wait 2 --no-clobber -I /2005,/2006,/2007,/2008,/2009,/2010,/2011,/2012,/2013,/2014,/2015,/2016,/2017 -X ""/*/*/comments"" https://24ways.org/archives/ If you leave this running for a few minutes, you’ll end up with a folder structure something like this: $ find 24ways.org 24ways.org 24ways.org/2013 24ways.org/2013/why-bother-with-accessibility 24ways.org/2013/why-bother-with-accessibility/index.html 24ways.org/2013/levelling-up 24ways.org/2013/levelling-up/index.html 24ways.org/2013/project-hubs 24ways.org/2013/project-hubs/index.html 24ways.org/2013/credits-and-recognition 24ways.org/2013/credits-and-recognition/index.html ... As a quick sanity check, let’s count the number of HTML pages we have retrieved: $ find 24ways.org | grep index.html | wc -l 328 There’s one last step! We got everything up to 2017, but we need to fetch the articles for 2018 (so far) as well. They aren’t linked in the /archives/ yet so we need to point our crawler at the site’s front page instead: wget --recursive --wait 2 --no-clobber -I /2018 -X ""/*/*/comments"" https://24ways.org/ Thanks to --no-clobber, this is safe to run every day in December to pick up any new content. We now have a folder on our computer containing an HTML file for every article that has ever been published on the site! Let’s use them to build ourselves a search index. Building a search index using SQLite There are many tools out there that can be used to build a search engine. You can use an open-source search server like Elasticsearch or Solr, a hosted option like Algolia or Amazon CloudSearch or you can tap into the built-in search features of relational databases like MySQL or PostgreSQL. I’m going to use something that’s less commonly used for web applications but makes for a powerful and extremely inexpensive alternative: SQLite. SQLite is the world’s most widely deployed database, even though many people have never even heard of it. That’s because it’s designed to be used as an embedded database: it’s commonly used by native mobile applications and even runs as part of the default set of apps on the Apple Watch! SQLite has one major limitation: unlike databases like MySQL and PostgreSQL, it isn’t really designed to handle large numbers of concurrent writes. For this reason, most people avoid it for building web applications. This doesn’t matter nearly so much if you are building a search engine for infrequently updated content - say one for a site that only publishes new content on 24 days every year. It turns out SQLite has very powerful full-text search functionality built into the core database - the FTS5 extension. I’ve been doing a lot of work with SQLite recently, and as part of that, I’ve been building a Python utility library to make building new SQLite databases as easy as possible, called sqlite-utils. It’s designed to be used within a Jupyter notebook - an enormously productive way of interacting with Python code that’s similar to the Observable notebooks Natalie described on 24 ways yesterday. If you haven’t used Jupyter before, here’s the fastest way to get up and running with it - assuming you have Python 3 installed on your machine. We can use a Python virtual environment to ensure the software we are installing doesn’t clash with any other installed packages: $ python3 -m venv ./jupyter-venv $ ./jupyter-venv/bin/pip install jupyter # ... lots of installer output # Now lets install some extra packages we will need later $ ./jupyter-venv/bin/pip install beautifulsoup4 sqlite-utils html5lib # And start the notebook web application $ ./jupyter-venv/bin/jupyter-notebook # This will open your browser to Jupyter at http://localhost:8888/ You should now be in the Jupyter web application. Click New -> Python 3 to start a new notebook. A neat thing about Jupyter notebooks is that if you publish them to GitHub (either in a regular repository or as a Gist), it will render them as HTML. This makes them a very powerful way to share annotated code. I’ve published the notebook I used to build the search index on my GitHub account. ​ Here’s the Python code I used to scrape the relevant data from the downloaded HTML files. Check out the notebook for a line-by-line explanation of what’s going on. from pathlib import Path from bs4 import BeautifulSoup as Soup base = Path(""/Users/simonw/Dropbox/Development/24ways-search"") articles = list(base.glob(""*/*/*/*.html"")) # articles is now a list of paths that look like this: # PosixPath('...24ways-search/24ways.org/2013/why-bother-with-accessibility/index.html') docs = [] for path in articles: year = str(path.relative_to(base)).split(""/"")[1] url = 'https://' + str(path.relative_to(base).parent) + '/' soup = Soup(path.open().read(), ""html5lib"") author = soup.select_one("".c-continue"")[""title""].split( ""More information about"" )[1].strip() author_slug = soup.select_one("".c-continue"")[""href""].split( ""/authors/"" )[1].split(""/"")[0] published = soup.select_one("".c-meta time"")[""datetime""] contents = soup.select_one("".e-content"").text.strip() title = soup.find(""title"").text.split("" ◆"")[0] try: topic = soup.select_one( '.c-meta a[href^=""/topics/""]' )[""href""].split(""/topics/"")[1].split(""/"")[0] except TypeError: topic = None docs.append({ ""title"": title, ""contents"": contents, ""year"": year, ""author"": author, ""author_slug"": author_slug, ""published"": published, ""url"": url, ""topic"": topic, }) After running this code, I have a list of Python dictionaries representing each of the documents that I want to add to the index. The list looks something like this: [ { ""title"": ""Why Bother with Accessibility?"", ""contents"": ""Web accessibility (known in other fields as inclus..."", ""year"": ""2013"", ""author"": ""Laura Kalbag"", ""author_slug"": ""laurakalbag"", ""published"": ""2013-12-10T00:00:00+00:00"", ""url"": ""https://24ways.org/2013/why-bother-with-accessibility/"", ""topic"": ""design"" }, { ""title"": ""Levelling Up"", ""contents"": ""Hello, 24 ways. Iu2019m Ashley and I sell property ins..."", ""year"": ""2013"", ""author"": ""Ashley Baxter"", ""author_slug"": ""ashleybaxter"", ""published"": ""2013-12-06T00:00:00+00:00"", ""url"": ""https://24ways.org/2013/levelling-up/"", ""topic"": ""business"" }, ... My sqlite-utils library has the ability to take a list of objects like this and automatically create a SQLite database table with the right schema to store the data. Here’s how to do that using this list of dictionaries. import sqlite_utils db = sqlite_utils.Database(""/tmp/24ways.db"") db[""articles""].insert_all(docs) That’s all there is to it! The library will create a new database and add a table to it called articles with the necessary columns, then insert all of the documents into that table. (I put the database in /tmp/ for the moment - you can move it to a more sensible location later on.) You can inspect the table using the sqlite3 command-line utility (which comes with OS X) like this: $ sqlite3 /tmp/24ways.db sqlite> .headers on sqlite> .mode column sqlite> select title, author, year from articles; title author year ------------------------------ ------------ ---------- Why Bother with Accessibility? Laura Kalbag 2013 Levelling Up Ashley Baxte 2013 Project Hubs: A Home Base for Brad Frost 2013 Credits and Recognition Geri Coady 2013 Managing a Mind Christopher 2013 Run Ragged Mark Boulton 2013 Get Started With GitHub Pages Anna Debenha 2013 Coding Towards Accessibility Charlie Perr 2013 ... There’s one last step to take in our notebook. We know we want to use SQLite’s full-text search feature, and sqlite-utils has a simple convenience method for enabling it for a specified set of columns in a table. We want to be able to search by the title, author and contents fields, so we call the enable_fts() method like this: db[""articles""].enable_fts([""title"", ""author"", ""contents""]) Introducing Datasette Datasette is the open-source tool I’ve been building that makes it easy to both explore SQLite databases and publish them to the internet. We’ve been exploring our new SQLite database using the sqlite3 command-line tool. Wouldn’t it be nice if we could use a more human-friendly interface for that? If you don’t want to install Datasette right now, you can visit https://search-24ways.herokuapp.com/ to try it out against the 24 ways search index data. I’ll show you how to deploy Datasette to Heroku like this later in the article. If you want to install Datasette locally, you can reuse the virtual environment we created to play with Jupyter: ./jupyter-venv/bin/pip install datasette This will install Datasette in the ./jupyter-venv/bin/ folder. You can also install it system-wide using regular pip install datasette. Now you can run Datasette against the 24ways.db file we created earlier like so: ./jupyter-venv/bin/datasette /tmp/24ways.db This will start a local webserver running. Visit http://localhost:8001/ to start interacting with the Datasette web application. If you want to try out Datasette without creating your own 24ways.db file you can download the one I created directly from https://search-24ways.herokuapp.com/24ways-ae60295.db Publishing the database to the internet One of the goals of the Datasette project is to make deploying data-backed APIs to the internet as easy as possible. Datasette has a built-in command for this, datasette publish. If you have an account with Heroku or Zeit Now, you can deploy a database to the internet with a single command. Here’s how I deployed https://search-24ways.herokuapp.com/ (running on Heroku’s free tier) using datasette publish: $ ./jupyter-venv/bin/datasette publish heroku /tmp/24ways.db --name search-24ways -----> Python app detected -----> Installing requirements with pip -----> Running post-compile hook -----> Discovering process types Procfile declares types -> web -----> Compressing... Done: 47.1M -----> Launching... Released v8 https://search-24ways.herokuapp.com/ deployed to Heroku If you try this out, you’ll need to pick a different --name, since I’ve already taken search-24ways. You can run this command as many times as you like to deploy updated versions of the underlying database. Searching and faceting Datasette can detect tables with SQLite full-text search configured, and will add a search box directly to the page. Take a look at http://search-24ways.herokuapp.com/24ways-b607e21/articles to see this in action. ​ SQLite search supports wildcards, so if you want autocomplete-style search where you don’t need to enter full words to start getting results you can add a * to the end of your search term. Here’s a search for access* which returns articles on accessibility: http://search-24ways.herokuapp.com/24ways-ae60295/articles?_search=acces%2A A neat feature of Datasette is the ability to calculate facets against your data. Here’s a page showing search results for svg with facet counts calculated against both the year and the topic columns: http://search-24ways.herokuapp.com/24ways-ae60295/articles?_search=svg&_facet=year&_facet=topic Every page visible via Datasette has a corresponding JSON API, which can be accessed using the JSON link on the page - or by adding a .json extension to the URL: http://search-24ways.herokuapp.com/24ways-ae60295/articles.json?_search=acces%2A Better search using custom SQL The search results we get back from ../articles?_search=svg are OK, but the order they are returned in is not ideal - they’re actually being returned in the order they were inserted into the database! You can see why this is happening by clicking the View and edit SQL link on that search results page. This exposes the underlying SQL query, which looks like this: select rowid, * from articles where rowid in ( select rowid from articles_fts where articles_fts match :search ) order by rowid limit 101 We can do better than this by constructing a custom SQL query. Here’s the query we will use instead: select snippet(articles_fts, -1, 'b4de2a49c8', '8c94a2ed4b', '...', 100) as snippet, articles_fts.rank, articles.title, articles.url, articles.author, articles.year from articles join articles_fts on articles.rowid = articles_fts.rowid where articles_fts match :search || ""*"" order by rank limit 10; You can try this query out directly - since Datasette opens the underling SQLite database in read-only mode and enforces a one second time limit on queries, it’s safe to allow users to provide arbitrary SQL select queries for Datasette to execute. There’s a lot going on here! Let’s break the SQL down line-by-line: select snippet(articles_fts, -1, 'b4de2a49c8', '8c94a2ed4b', '...', 100) as snippet, We’re using snippet(), a built-in SQLite function, to generate a snippet highlighting the words that matched the query. We use two unique strings that I made up to mark the beginning and end of each match - you’ll see why in the JavaScript later on. articles_fts.rank, articles.title, articles.url, articles.author, articles.year These are the other fields we need back - most of them are from the articles table but we retrieve the rank (representing the strength of the search match) from the magical articles_fts table. from articles join articles_fts on articles.rowid = articles_fts.rowid articles is the table containing our data. articles_fts is a magic SQLite virtual table which implements full-text search - we need to join against it to be able to query it. where articles_fts match :search || ""*"" order by rank limit 10; :search || ""*"" takes the ?search= argument from the page querystring and adds a * to the end of it, giving us the wildcard search that we want for autocomplete. We then match that against the articles_fts table using the match operator. Finally, we order by rank so that the best matching results are returned at the top - and limit to the first 10 results. How do we turn this into an API? As before, the secret is to add the .json extension. Datasette actually supports multiple shapes of JSON - we’re going to use ?_shape=array to get back a plain array of objects: JSON API call to search for articles matching SVG The HTML version of that page shows the time taken to execute the SQL in the footer. Hitting refresh a few times, I get response times between 2 and 5ms - easily fast enough to power a responsive autocomplete feature. A simple JavaScript autocomplete search interface I considered building this using React or Svelte or another of the myriad of JavaScript framework options available today, but then I remembered that vanilla JavaScript in 2018 is a very productive environment all on its own. We need a few small utility functions: first, a classic debounce function adapted from this one by David Walsh: function debounce(func, wait, immediate) { let timeout; return function() { let context = this, args = arguments; let later = () => { timeout = null; if (!immediate) func.apply(context, args); }; let callNow = immediate && !timeout; clearTimeout(timeout); timeout = setTimeout(later, wait); if (callNow) func.apply(context, args); }; }; We’ll use this to only send fetch() requests a maximum of once every 100ms while the user is typing. Since we’re rendering data that might include HTML tags (24 ways is a site about web development after all), we need an HTML escaping function. I’m amazed that browsers still don’t bundle a default one of these: const htmlEscape = (s) => s.replace( />/g, '>' ).replace( /Autocomplete search

And now the autocomplete implementation itself, as a glorious, messy stream-of-consciousness of JavaScript: // Embed the SQL query in a multi-line backtick string: const sql = `select snippet(articles_fts, -1, 'b4de2a49c8', '8c94a2ed4b', '...', 100) as snippet, articles_fts.rank, articles.title, articles.url, articles.author, articles.year from articles join articles_fts on articles.rowid = articles_fts.rowid where articles_fts match :search || ""*"" order by rank limit 10`; // Grab a reference to the const searchbox = document.getElementById(""searchbox""); // Used to avoid race-conditions: let requestInFlight = null; searchbox.onkeyup = debounce(() => { const q = searchbox.value; // Construct the API URL, using encodeURIComponent() for the parameters const url = ( ""https://search-24ways.herokuapp.com/24ways-866073b.json?sql="" + encodeURIComponent(sql) + `&search=${encodeURIComponent(q)}&_shape=array` ); // Unique object used just for race-condition comparison let currentRequest = {}; requestInFlight = currentRequest; fetch(url).then(r => r.json()).then(d => { if (requestInFlight !== currentRequest) { // Avoid race conditions where a slow request returns // after a faster one. return; } let results = d.map(r => `

${htmlEscape(r.title)}

${htmlEscape(r.author)} - ${r.year}

${highlight(r.snippet)}

`).join(""""); document.getElementById(""results"").innerHTML = results; }); }, 100); // debounce every 100ms There’s just one more utility function, used to help construct the HTML results: const highlight = (s) => htmlEscape(s).replace( /b4de2a49c8/g, '' ).replace( /8c94a2ed4b/g, '' ); This is what those unique strings passed to the snippet() function were for. Avoiding race conditions in autocomplete One trick in this code that you may not have seen before is the way race-conditions are handled. Any time you build an autocomplete feature, you have to consider the following case: User types acces Browser sends request A - querying documents matching acces* User continues to type accessibility Browser sends request B - querying documents matching accessibility* Request B returns. It was fast, because there are fewer documents matching the full term The results interface updates with the documents from request B, matching accessibility* Request A returns results (this was the slower of the two requests) The results interface updates with the documents from request A - results matching access* This is a terrible user experience: the user saw their desired results for a brief second, and then had them snatched away and replaced with those results from earlier on. Thankfully there’s an easy way to avoid this. I set up a variable in the outer scope called requestInFlight, initially set to null. Any time I start a new fetch() request, I create a new currentRequest = {} object and assign it to the outer requestInFlight as well. When the fetch() completes, I use requestInFlight !== currentRequest to sanity check that the currentRequest object is strictly identical to the one that was in flight. If a new request has been triggered since we started the current request we can detect that and avoid updating the results. It’s not a lot of code, really And that’s the whole thing! The code is pretty ugly, but when the entire implementation clocks in at fewer than 70 lines of JavaScript, I honestly don’t think it matters. You’re welcome to refactor it as much you like. How good is this search implementation? I’ve been building search engines for a long time using a wide variety of technologies and I’m happy to report that using SQLite in this way is genuinely a really solid option. It scales happily up to hundreds of MBs (or even GBs) of data, and the fact that it’s based on SQL makes it easy and flexible to work with. A surprisingly large number of desktop and mobile applications you use every day implement their search feature on top of SQLite. More importantly though, I hope that this demonstrates that using Datasette for an API means you can build relatively sophisticated API-backed applications with very little backend programming effort. If you’re working with a small-to-medium amount of data that changes infrequently, you may not need a more expensive database. Datasette-powered applications easily fit within the free tier of both Heroku and Zeit Now. For more of my writing on Datasette, check out the datasette tag on my blog. And if you do build something fun with it, please let me know on Twitter.",2018,Simon Willison,simonwillison,2018-12-19T00:00:00+00:00,https://24ways.org/2018/fast-autocomplete-search-for-your-website/,code 123,Fast and Simple Usability Testing,"Everyone knows by now that they should test the usability of their applications, but still hardly anybody actually does it. In this article I’ll share some tips I’ve picked up for doing usability tests quickly and effectively. Relatively recent tools like Django and Ruby on Rails allow us to develop projects faster and to make significant changes later in the project timeline. Usability testing methods should now be adapted to fit this modern approach to development. When to test In an ideal world usability tests would be carried out frequently from an early stage of the project. Time and budget constraints lead this to be impractical; usability is often the first thing to get dropped from the project plan. If you can only test at one stage in the project, whatever the size, the most valuable time is before your first public beta — leaving long enough to fix issues and not so late that you can’t rethink your scope. There are three main categories of usability test: Testing design mockups Testing a new working application Testing established applications Each category requires a slightly different approach. For small modern web projects you are most likely to be testing a new working application. You will of course have already done functional tests so you won’t be worried about the user breaking things. The main differences between the categories apply in how you word The Script. Testing an established application is the most fun in my opinion. Humans are remarkably adaptable and rapidly develop coping strategies to work around usability issues in software they are forced to use. Uncovering these strategies may lead you to understand previously unspoken needs of your users. Often small changes to the application will have a dramatic affect on their everyday lives. Who to test When you have built a project to scratch your own itch, your intended audience will be people just like you. Test subjects in this case should be easy to find – friends, co-workers etc. This is not always the case; your users may not be like you at all. When they are not, it’s all the more important to run usability tests. Testing on friends, family and co-workers is better than not doing usability tests at all, but it can’t be compared to testing on actual samples of your intended audience. People who would use the system will provide more genuine feedback and deeper insight. Never let your test subjects put themselves in the shoes of your ‘actual’ users. For example, you should discourage comments like “Well, I would do this BUT if I was a bus driver I’d do that”. Users are not qualified to put themselves in the position of others. Inaccurate data is often worse than no data. Aim for five or six test subjects: any more and you probably won’t learn anything new; any less and you’re likely to be overwhelmed by issues stemming from people’s individual personalities. The Script The Script is a single side of A4 (or letter) paper, consisting of questions for your testers and reminders for yourself. Have a balance of task-based questions and expectation analysis. This helps maintain consistency across tests. Expectation analysis is more important for testing designs and new applications: “Where would you find X?”, “What would you expect to happen if you clicked on Y?”. In an established system users will probably know where these things are though it can still be illuminating to ask these questions though phrased slightly differently. Task-based questions involve providing a task for the user to complete. If you are testing an established system it is a good idea to ask users to bring in tasks that they would normally perform. This is because the user will be more invested in the outcome of the task and will behave in a more realistic fashion. When designing tasks for new systems and designs ensure you only provide loose task details for the same reason. Don’t tell testers to enter “Chantelle”; have them use their own name instead. Avoid introducing bias with the way questions are phrased. It’s a good idea to ask for users’ first impressions at the beginning of the test, especially when testing design mockups. “What are the main elements on the page?” or “What strikes you first?”. You script should run for a maximum of 45 minutes. 30-35 minutes is better; after this you are likely to lose their attention. Tests on established systems can take longer as there is more to learn from them. When scheduling the test you will need to leave yourself 5 minutes between each one to collate your notes and prepare for the next. Be sure to run through the script beforehand. Your script should be flexible. It is possible that during the test a trend will come to light that opens up whole new avenues of possible questioning. For example, during one initial test of an established system I noticed that the test subject had been printing off items from the application and placing them in a folder in date order (the system ordered alphabetically). I changed the script to ask future participants in that run, if they ever used external tools to help them with tasks within the system. This revealed a number of interesting issues that otherwise would not have been found. Running the tests Treat your test subjects like hedgehogs. Depending on your target audience they probably feel a little nervous and perhaps even scared of you. So make them a little nest out of straw, stroke their prickles and give them some cat food. Alternatively, reassure them that you are testing the system and that they can’t give a wrong answer. Reward them with a doughnut or jam tart at the end. Try to ensure the test environment is relaxed and quiet, but also as close as possible to the situation where they would actually use the system. Have your subjects talk out loud is very important as you can’t read their minds, but it is a very unnatural process. To loosen up your subjects and get them talking in the way you want them to, try the Stapler Trick. Give them a stapler or similar item and ask them to open it, take the staples out, replace them, shut the stapler and staple some paper – talking all the time about what they see, what they expect to happen, what actually happens and how that matches up. Make them laugh at you. Say how long the test will take up front, and tell your subject why you are doing it. After the test has been completed, conclude by thanking them for their time and assuring them that they were very useful. Then give them the sugary treat. What to look for Primarily, you should look out for incidents where the user stops concentrating on her tasks and starts thinking about the tool and how she is going to use it. For example, when you are hammering in a nail you don’t think about how to use a hammer; good software should be the same. Words like ‘it’ and ‘the system’ and are good indications that the test subject has stopped thinking about the task in hand. Note questioning words, especially where testers question their own judgement, “why can’t I find …”, “I expected to see …” etc. as this indicates that the work flow for the task may have broken down. Also keep an eye on occasions where the user completely fails to do a task. They may need some prompting to unstick them, but you should be careful not to bias the test. These should be the highest priority issues for you to fix. If users recover from getting stuck, make a note of how they recovered. Prolonged periods of silence from the test subject may also require prompting as they should be talking all the time. Ask them what they are thinking or looking for but avoid words like ‘try’ (e.g. ‘what are you trying to do?’) as this implies that they are currently failing. Be wary of users’ opinions on aesthetics and be prepared to bring them back to the script if they get side-tracked. Writing it up Even if you are the only developer it’s important to summarise the key issues that emerged during testing: your notes won’t make much sense to you a week or so after the test. If you are writing for other people, include a summary no longer than two pages; this can consist of a list or table of the issues including recommendations and their priorities. Remember to anonymise the users in the report. In team situations, you may be surprised at how many people are interested in the results of the usability test even if it doesn’t relate directly to something that they can fix. To conclude… Some usability testing is better than none at all, even for small projects or those with strict deadlines. Make the most of the time and resources available. Choose your users carefully, make them comfortable, summarise your report and don’t forget to leave a doughnut for yourself!",2006,Natalie Downe,nataliedowne,2006-12-16T00:00:00+00:00,https://24ways.org/2006/fast-and-simple-usability-testing/,process 130,Faster Development with CSS Constants,"Anyone even slightly familiar with a programming language will have come across the concept of constants – a fixed value that can be used through your code. For example, in a PHP script I might have a constant which is the email address that all emails generated by my application get sent to. $adminEmail = 'info@example.com'; I could then use $adminEmail in my script whenever I wanted an email to go to that address. The benefit of this is that when the client decides they want the email to go to a different address, I only need change it in one place – the place where I initially set the constant. I could also quite easily make this value user defined and enable the administrator to update the email address. Unfortunately CSS doesn’t support constants. It would be really useful to be able to define certain values initially and then use them throughout a CSS file, so in this article I’m going to take a look at some of the methods we do have available and provide pointers to more in depth commentary on each. If you have a different method, or tip to share please add it to the comments. So what options do we have? One way to get round the lack of constants is to create some definitions at the top of your CSS file in comments, to define ‘constants’. A common use for this is to create a ‘color glossary’. This means that you have a quick reference to the colors used in the site to avoid using alternates by mistake and, if you need to change the colors, you have a quick list to go down and do a search and replace. In the below example, if I decide I want to change the mid grey to #999999, all I need to do is search and replace #666666 with #999999 – assuming I’ve remember to always use that value for things which are mid grey. /* Dark grey (text): #333333 Dark Blue (headings, links) #000066 Mid Blue (header) #333399 Light blue (top navigation) #CCCCFF Mid grey: #666666 */ This is a fairly low-tech method, but if used throughout the development of the CSS files can make changes far simpler and help to ensure consistency in your color scheme. I’ve seen this method used by many designers however Garrett Dimon documents the method, with more ideas in the comments. Going server-side To truly achieve constants you will need to use something other than CSS to process the file before it is sent to the browser. You can use any scripting language – PHP, ASP, ColdFusion etc. to parse a CSS file in which you have entered constants. So that in a constants section of the CSS file you would have: $darkgrey = '#333333'; $darkblue = '#000066'; The rest of the CSS file is as normal except that when you come to use the constant value you would use the constant name instead of adding the color: p { color: $darkgrey; } Your server-side script could then parse the CSS file, replace the constant names with the constant values and serve a valid CSS file to the browser. Christian Heilmann has done just this for PHP however this could be adapted for any language you might have available on your server. Shaun Inman came up with another way of doing this that removes the need to link to a PHP script and also enables the adding of constants using the syntax of at-rules . This method is again using PHP and will require you to edit an .htaccess file. A further method is to generate static CSS files either using a script locally – if the constants are just to enable speed of development – or as part of the web application itself. Storing a template stylesheet with constant names in place of the values you will want to update means that your script can simply open the template, replace the variables and save the result as a new stylesheet file. While CSS constants are a real help to developers, they can also be used to add new functionality to your applications. As with the email address example that I used at the beginning of this article, using a combination of CSS and server-side scripting you could enable a site administrator to select the colours for a new theme to be used on a page of a content managed site. By using constants you need only give them the option to change certain parts of the CSS and not upload a whole different CSS file, which could lead to some interesting results! As we are unlikely to find real CSS constants under the tree this Christmas the above methods are some possibilities for better management of your stylesheets. However if you have better methods, CSS Constant horror stories or any other suggestions, add your comments below.",2006,Rachel Andrew,rachelandrew,2006-12-02T00:00:00+00:00,https://24ways.org/2006/faster-development-with-css-constants/,process 209,Feeding the Audio Graph,"In 2004, I was given an iPod. I count this as one of the most intuitive pieces of technology I’ve ever owned. It wasn’t because of the the snazzy (colour!) menus or circular touchpad. I loved how smoothly it fitted into my life. I could plug in my headphones and listen to music while I was walking around town. Then when I got home, I could plug it into an amplifier and carry on listening there. There was no faff. It didn’t matter if I could find my favourite mix tape, or if my WiFi was flakey - it was all just there. Nowadays, when I’m trying to pair my phone with some Bluetooth speakers, or can’t find my USB-to-headphone jack, or even access any music because I don’t have cellular reception; I really miss this simplicity. The Web Audio API I think the Web Audio API feels kind of like my iPod did. It’s different from most browser APIs - rather than throwing around data, or updating DOM elements - you plug together a graph of audio nodes, which the browser uses to generate, process, and play sounds. The thing I like about it is that you can totally plug it into whatever you want, and it’ll mostly just work. So, let’s get started. First of all we want an audio source.