Planet Firefox

October 30, 2014

Tim Taubert (ttaubert)

Why including a backup pin in your Public-Key-Pinning header is a good idea

In my last post “Deploying TLS the hard way” I explained how TLS and its extensions (as well as a few HTTP extensions) work and what to watch out for when enabling TLS for your server. One of the HTTP extensions mentioned is HTTP Public-Key-Pinning (HPKP). As a short reminder, the header looks like this:

Public-Key-Pins:
  pin-sha256="GRAH5Ex+kB4cCQi5gMU82urf+6kEgbVtzfCSkw55AGk=";
  pin-sha256="lERGk61FITjzyKHcJ89xpc6aDwtRkOPAU0jdnUqzW2s=";
  max-age=15768000; includeSubDomains

You can see that it specifies two pin-sha256 values, that is the pins of two public keys. One is the public key of your currently valid certificate and the other is a backup key in case you have to revoke your certificate.

I received a few questions as to why I suggest including a backup pin and what the requirements for a backup key would be. I will try to answer those with a more detailed overview of how public key pinning and TLS certificates work.

How are RSA keys represented?

Let us go back to the beginning and start by taking a closer look at RSA keys:

$ openssl genrsa 4096

The above command generates a 4096 bit RSA key and prints it to the console. Although it says -----BEGIN RSA PRIVATE KEY----- it does not only return the private key but an ASN.1 structure that also contains the public key - we thus actually generated an RSA key pair.

A common misconception when learning about keys and certificates is that the RSA key itself for a given certificate expires. RSA keys however never expire - after all they are just three numbers. Only the certificate containing the public key can expire and only the certificate can be revoked. Keys “expire” or are “revoked” as soon as there are no more valid certificates using the public key, and you threw away the keys and stopped using them altogether.

What does the TLS certificate contain?

By submitting the Certificate Signing Request (CSR) containing your public key to a Certificate Authority it will issue a valid certificate. That will again contain the public key of the RSA key pair we generated above and an expiration date. Both the public key and the expiration date will be signed by the CA so that modifications of any of the two would render the certificate invalid immediately.

For simplicity I left out a few other fields that X.509 certificates contain to properly authenticate TLS connections, for example your server’s hostname and other details.

How does public key pinning work?

The whole purpose of public key pinning is to detect when the public key of a certificate for a specific host has changed. That may happen when an attacker compromises a CA such that they are able to issue valid certificates for any domain. A foreign CA might also just be the attacker, think of state-owned CAs that you do not want to be able to {M,W}ITM your site. Any attacker intercepting a connection from a visitor to your server with a forged certificate can only be prevented by detecting that the public key has changed.

After the server sent a TLS certificate with the handshake, the browser will look up any stored pins for the given hostname and check whether any of those stored pins match any of the SPKI fingerprints (the output of applying SHA-256 to the public key information) in the certificate chain. The connection must be terminated immediately if pin validation fails.

If the browser does not find any stored pins for the current hostname then it will directly continue with the usual certificate checks. This might happen if the site does not support public key pinning and does not send any HPKP headers at all, or if this is the first time visiting and the server has not seen the HPKP header yet in a previous visit.

Pin validation should happen as soon as possible and thus before any basic certificate checks are performed. An expired or revoked certificate will be happily accepted at the pin validation stage early in the handshake when any of the SPKI fingerprints of its chain match a stored pin. Only a little later the browser will see that the certificate already expired or was revoked and will reject it.

Pin validation also works for self-signed certificates, but they will of course raise the same warnings as usual as soon as the browser determined they were not signed by a trusted third-party.

What if your certificate was revoked?

If your server was compromised and an attacker obtained your private key you have to revoke your certificate as the attacker can obviously fully intercept any TLS connection to your server and record every conversation. If your HPKP header contained only a single pin-sha256 token you are out of luck until the max-age directive given in the header lets those pins expire in your visitors’ browsers.

Pin validation requires checking the SPKI fingerprints of all certificates in the chain. When for example StartSSL signed your certificate you have another intermediate Class 1 or 2 certificate and their root certificate in the chain. The browser trusts only the root certificate but the intermediate ones are signed by the root certificate. The intermediate certificate in turn signs the certificate deployed on your server and that is called a chain of trust.

To prevent getting stuck after your only pinned key was compromised, you could for example provide the SPKI fingerprint of StartSSL’s Class 1 intermediate certificate. An attacker would now have to somehow get a certificate issued by StartSSL’s Class 1 tier to successfully impersonate you. You are however again out of luck should you decide to upgrade to Class 2 in a month because you decided to start paying for a certificate.

Pinning StartSSL’s root certificate would let you switch Classes any time and the attacker would still have to get a certificate issued by StartSSL for your domain. This is a valid approach as long as you are trusting your CA (really?) and as long as the CA itself is not compromised. In case of a compromise however the attacker would be able to get a valid certificate for your domain that passes pin validation. After the attack was discovered StartSSL would quickly revoke all currently issued certificates, generate a new key pair for their root certificate and issue new certificates. And again we would be out of luck because suddenly pin validation fails and no browser will connect to our site.

Include the pin of a backup key

The safest way to pin your certificate’s public key and be prepared to revoke your certificate when necessary is to include the pin of a second public key: your backup key. This backup RSA key should in no way be related to your first key, just generate a new one.

A good advice is to keep this backup key pair (especially the private key) in a safe place until you need it. Uploading it to the server is dangerous: when your server is compromised you lose both keys at once and have no backup key left.

Generate a pin for the backup key exactly as you did for the current key and include both pin-sha256 values as shown above in the HPKP header. In case the current key is compromised make sure all vulnerabilities are patched and then remove the revoked pin. Generate a CSR for the backup key, let your CA issue a new certificate, and revoke the old one. Upload the new certificate to your server and you are done.

Finally, do not forget to generate a new backup key and include that pin in your HPKP header again. Once a browser successfully establishes a TLS connection the next time, it will see your updated HPKP header and replace any stored pins with the new ones.

October 30, 2014 01:00 PM

HTTP Public-Key-Pinning explained

In my last post “Deploying TLS the hard way” I explained how TLS and its extensions (as well as a few HTTP extensions) work and what to watch out for when enabling TLS for your server. One of the HTTP extensions mentioned is HTTP Public-Key-Pinning (HPKP). As a short reminder, the header looks like this:

Public-Key-Pins:
  pin-sha256="GRAH5Ex+kB4cCQi5gMU82urf+6kEgbVtzfCSkw55AGk=";
  pin-sha256="lERGk61FITjzyKHcJ89xpc6aDwtRkOPAU0jdnUqzW2s=";
  max-age=15768000; includeSubDomains

You can see that it specifies two pin-sha256 values, that is the pins of two public keys. One is the public key of your currently valid certificate and the other is a backup key in case you have to revoke your certificate.

I received a few questions as to why I suggest including a backup pin and what the requirements for a backup key would be. I will try to answer those with a more detailed overview of how public key pinning and TLS certificates work.

How are RSA keys represented?

Let us go back to the beginning and start by taking a closer look at RSA keys:

$ openssl genrsa 4096

The above command generates a 4096 bit RSA key and prints it to the console. Although it says -----BEGIN RSA PRIVATE KEY----- it does not only return the private key but an ASN.1 structure that also contains the public key - we thus actually generated an RSA key pair.

A common misconception when learning about keys and certificates is that the RSA key itself for a given certificate expires. RSA keys however never expire - after all they are just numbers. Only the certificate containing the public key can expire and only the certificate can be revoked. Keys “expire” or are “revoked” as soon as there are no more valid certificates using the public key, and you threw away the keys and stopped using them altogether.

What does the certificate contain?

By submitting the Certificate Signing Request (CSR) containing your public key to a Certificate Authority it will issue a valid certificate. That will again contain the public key of the RSA key pair we generated above and an expiration date. Both the public key and the expiration date will be signed by the CA so that modifications of any of the two would render the certificate invalid immediately.

For simplicity I left out a few other fields that X.509 certificates contain to properly authenticate TLS connections, for example your server’s hostname and other details.

How does public key pinning work?

The whole purpose of public key pinning is to detect when the public key of a certificate for a specific host has changed. That may happen when an attacker compromises a CA such that they are able to issue valid certificates for any domain. A foreign CA might also just be the attacker, think of state-owned CAs that you do not want to be able to {M,W}ITM your site. Any attacker intercepting a connection from a visitor to your server with a forged certificate can only be prevented by detecting that the public key has changed.

After the server sent a TLS certificate with the handshake, the browser will look up any stored pins for the given hostname and check whether any of those stored pins match any of the SPKI fingerprints (the output of applying SHA-256 to the public key information) in the certificate chain. The connection must be terminated immediately if pin validation fails.

If the browser does not find any stored pins for the current hostname then it will directly continue with the usual certificate checks. This might happen if the site does not support public key pinning and does not send any HPKP headers at all, or if this is the first time visiting and the server has not seen the HPKP header yet in a previous visit.

Pin validation should happen as soon as possible and thus before any basic certificate checks are performed. An expired or revoked certificate will be happily accepted at the pin validation stage early in the handshake when any of the SPKI fingerprints of its chain match a stored pin. Only a little later the browser will see that the certificate already expired or was revoked and will reject it.

Pin validation also works for self-signed certificates, but they will of course raise the same warnings as usual as soon as the browser determined they were not signed by a trusted third-party.

What if your certificate was revoked?

If your server was compromised and an attacker obtained your private key you have to revoke your certificate as the attacker can obviously fully intercept any TLS connection to your server and record every conversation. If your HPKP header contained only a single pin-sha256 token you are out of luck until the max-age directive given in the header lets those pins expire in your visitors’ browsers.

Pin validation requires checking the SPKI fingerprints of all certificates in the chain. When for example StartSSL signed your certificate you have another intermediate Class 1 or 2 certificate and their root certificate in the chain. The browser trusts only the root certificate but the intermediate ones are signed by the root certificate. The intermediate certificate in turn signs the certificate deployed on your server and that is called a chain of trust.

To prevent getting stuck after your only pinned key was compromised, you could for example provide the SPKI fingerprint of StartSSL’s Class 1 intermediate certificate. An attacker would now have to somehow get a certificate issued by StartSSL’s Class 1 tier to successfully impersonate you. You are however again out of luck should you decide to upgrade to Class 2 in a month because you decided to start paying for a certificate.

Pinning StartSSL’s root certificate would let you switch Classes any time and the attacker would still have to get a certificate issued by StartSSL for your domain. This is a valid approach as long as you are trusting your CA (really?) and as long as the CA itself is not compromised. In case of a compromise however the attacker would be able to get a valid certificate for your domain that passes pin validation. After the attack was discovered StartSSL would quickly revoke all currently issued certificates, generate a new key pair for their root certificate and issue new certificates. And again we would be out of luck because suddenly pin validation fails and no browser will connect to our site.

Include the pin of a backup key

The safest way to pin your certificate’s public key and be prepared to revoke your certificate when necessary is to include the pin of a second public key: your backup key. This backup RSA key should in no way be related to your first key, just generate a new one.

A good advice is to keep this backup key pair (especially the private key) in a safe place until you need it. Uploading it to the server is dangerous: when your server is compromised you lose both keys at once and have no backup key left.

Generate a pin for the backup key exactly as you did for the current key and include both pin-sha256 values as shown above in the HPKP header. In case the current key is compromised make sure all vulnerabilities are patched and then remove the revoked pin. Generate a CSR for the backup key, let your CA issue a new certificate, and revoke the old one. Upload the new certificate to your server and you are done.

Finally, do not forget to generate a new backup key and include that pin in your HPKP header again. Once a browser successfully establishes a TLS connection the next time, it will see your updated HPKP header and replace any stored pins with the new ones.

October 30, 2014 01:00 PM

October 29, 2014

Tim Taubert (ttaubert)

Talk: Keeping secrets with JavaScript - An Introduction to the WebCrypto API

With the web slowly maturing as a platform the demand for cryptography in the browser has risen, especially in a post-Snowden era. Many of us have heard about the upcoming Web Cryptography API but at the time of writing there seem to be no good introductions available. We will take a look at the proposed W3C spec and its current state of implementation.

Video

Slides

Code

https://github.com/ttaubert/secret-notes

October 29, 2014 04:00 PM

October 27, 2014

Tim Taubert (ttaubert)

Deploying TLS the hard way

  1. How does TLS work?
  2. The certificate
  3. (Perfect) Forward Secrecy
  4. Choosing the right cipher suites
  5. HTTP Strict Transport Security
  6. HSTS Preload List
  7. OCSP Stapling
  8. HTTP Public Key Pinning
  9. Known attacks

Last weekend I finally deployed TLS for timtaubert.de and decided to write up what I learned on the way hoping that it would be useful for anyone doing the same. Instead of only giving you a few buzz words I want to provide background information on how TLS and certain HTTP extensions work and why you should use them or configure TLS in a certain way.

One thing that bugged me was that most posts only describe what to do but not necessarily why to do it. I hope you appreciate me going into a little more detail to end up with the bigger picture of what TLS currently is, so that you will be able to make informed decisions when deploying yourselves.

To follow this post you will need some basic cryptography knowledge. Whenever you do not know or understand a concept you should probably just head over to Wikipedia and take a few minutes or just do it later and maybe re-read the whole thing.

Disclaimer: I am not a security expert or cryptographer but did my best to research this post thoroughly. Please let me know of any mistakes I might have made and I will correct them as soon as possible.

But didn’t Andy say this is all shit?

I read Andy Wingo’s blog post too and I really liked it. Everything he says in there is true. But what is also true is that TLS with the few add-ons is all we have nowadays and we better make the folks working for the NSA earn their money instead of not trying to encrypt traffic at all.

After you finished reading this page, maybe go back to Andy’s post and read it again. You might have a better understanding of what he is ranting about than you had before if the details of TLS are still dark matter to you.

So how does TLS work?

Every TLS connection starts with both parties sharing their supported TLS versions and cipher suites. As the next step the server sends its X.509 certificate to the browser.

Checking the server’s certificate

The following certificate checks need to be performed:

All of these are very obvious crucial checks. To query a certificate’s revocation status the browser will use the Online Certificate Status Protocol (OCSP) which I will describe in more detail in a later section.

After the certificate checks are done and the browser ensured it is talking to the right host both sides need to agree on secret keys they will use to communicate with each other.

Key Exchange using RSA

A simple key exchange would be to let the client generate a master secret and encrypt that with the server’s public RSA key given by the certificate. Both client and server would then use that master secret to derive symmetric encryption keys that will be used throughout this TLS session. An attacker could however simply record the handshake and session for later, when breaking the key has become feasible or the machine is suspect to a vulnerability. They may then use the server’s private key to recover the whole conversation.

Key Exchange using (EC)DHE

When using (Elliptic Curve) Diffie-Hellman as the key exchange mechanism both sides have to collaborate to generate a master secret. They generate DH key pairs (which is a lot cheaper than generating RSA keys) and send their public key to the other party. With the private key and the other party’s public key the shared master secret can be calculated and then again be used to derive session keys. We can provide Forward Secrecy when using ephemeral DH key pairs. See the section below on how to enable it.

We could in theory also provide forward secrecy with an RSA key exchange if the server would generate an ephemeral RSA key pair, share its public key and would then wait for the master secret to be sent by the client. As hinted above RSA key generation is very expensive and does not scale in practice. That is why RSA key exchanges are not a practical option for providing forward secrecy.

After both sides have agreed on session keys the TLS handshake is done and they can finally start to communicate using symmetric encryption algorithms like AES that are much faster than asymmetric algorithms.

The certificate

Now that we understand authenticity is an integral part of TLS we know that in order to serve a site via TLS we first need a certificate. The TLS protocol can encrypt traffic between two parties just fine but the certificate provides the necessary authentication towards visitors.

Without a certificate a visitor could securely talk to either us, the NSA, or a different attacker but they probably want to talk to us. The certificate ensures by cryptographic means that they established a connection to our server.

Selecting a Certificate Authority (CA)

If you want a cheap certificate, have no specific needs, and only a single subdomain (e.g. www) then StartSSL is an easy option. Do of course feel free to take a look at different authorities - their services and prices will vary heavily.

In the chain of trust the CA plays an important role: by verifying that you are the rightful owner of your domain and signing your certificate it will let browsers trust your certificate. The browsers do not want to do all this verification themselves so they defer it to the CAs.

For your certificate you will need an RSA key pair, a public and private key. The public key will be included in your certificate and thus also signed by the CA.

Generating an RSA key and a certificate signing request

The example below shows how you can use OpenSSL on the command line to generate a key for your domain. Simply replace example.com with the domain of your website. example.com.key will be your new RSA key and example.com.csr will be the Certificate Signing Request that your CA needs to generate your certificate.

openssl req -new -newkey rsa:4096 -nodes -sha256 \
  -keyout example.com.key -out example.com.csr

We will use a SHA-256 based signature for integrity as Firefox and Chrome will phase out support for SHA-1 based certificates soon. The RSA keys used to authenticate your website will use a 4096 bit modulus. If you need to handle a lot of traffic or your server has a weak CPU you might want to use 2048 bit. Never go below that as keys smaller than 2048 bit are considered insecure.

Get a signed certificate

Sign up with the CA you chose and depending on how they handle this process you probably will have to first verify that you are the rightful owner of the domain that you claim to possess. StartSSL will do that by sending a token to postmaster@example.com (or similar) and then ask you to confirm the receipt of that token.

Now that you signed up and are the verified owner of example.com you simply submit the example.com.csr file to request the generation of a certificate for your domain. The CA will sign your public key and the other information contained in the CSR with their private key and you can finally download the certificate to example.com.crt.

Upload the .crt and .key files to your web server. Be aware that any intermediate certificate in the CA’s chain must be included in the .crt file as well - you can just cat them together. StartSSL’s free tier has an intermediate Class 1 certificate - make sure to use the SHA-256 version of it. All files should be owned by root and must not be readable by anyone else. Configure your web server to use those and you should probably have TLS running configured out-of-the-box.

(Perfect) Forward Secrecy

To properly deploy TLS you will want to provide (Perfect) Forward Secrecy. Without forward secrecy TLS still seems to secure your communication today, it might however not if your private key is compromised in the future.

If a powerful adversary (think NSA) records all communication between a visitor and your server, they can decrypt all this traffic years later by stealing your private key or going the “legal” way to obtain it. This can be prevented by using short-lived (ephemeral) keys for key exchanges that the server will throw away after a short period.

Diffie-Hellman key exchanges

Using RSA with your certificate’s private and public keys for key exchanges is off the table as generating a 2048+ bit prime is very expensive. We thus need to switch to ephemeral (Elliptic Curve) Diffie-Hellman cipher suites. For DH you can generate a 2048 bit parameter once, choosing a private key afterwards is cheap.

openssl dhparam -out dhparam.pem 2048

Simply upload dhparam.pem to your server and instruct the web server to use it for Diffie-Hellman key exchanges. When using ECDH the predefined elliptic curve represents this parameter and no further action is needed.

(Nginx)
ssl_dhparam /path/to/ssl/dhparam.pem;

Apache does unfortunately not support custom DH parameters, it is always set to 1024 bit and is not user configurable. This might hopefully be fixed in future versions.

Session IDs

One of the most important mechanisms to improve TLS performance is Session Resumption. In a full handshake the server sends a Session ID as part of the “hello” message. On a subsequent connection the client can use this session ID and pass it to the server when connecting. Because both the server and the client have saved the last session’s “secret state” under the session ID they can simply resume the TLS session where they left off.

Now you might notice that this could violate forward secrecy as a compromised server might reveal the secret state for all session IDs if the cache is just large enough. The forward secrecy of a connection is thus bounded by how long the session information is retained on the server. Ideally, your server would use a medium-sized in-memory cache that is purged daily.

Apache lets you configure that using the SSLSessionCache directive and you should use the high-performance cyclic buffer shmcd. Nginx has the ssl_session_cache directive and you should use a shared cache that is shared between workers. The right size of those caches would depend on the amount of traffic your server handles. You want browsers to resume TLS sessions but also get rid of old ones about daily.

Session Tickets

The second mechanism to resume a TLS session are Session Tickets. This extension transmits the server’s secret state to the client, encrypted with a key only known to the server. That ticket key is protecting the TLS connection now and in the future.

This might as well violate forward secrecy if the key used to encrypt session tickets is compromised. The ticket (just as the session cache) contains all of the server’s secret state and would allow an attacker to reveal the whole conversation.

Nginx and Apache by default generate a session ticket key at startup and do unfortunately provide no way to rotate it. If your server is running for months without a restart then you will use that same session ticket key for months and breaking into your server could reveal every recorded TLS conversation since the web server was started.

Neither Nginx nor Apache have a sane way to work around this, Nginx might be able to rotate the key by reloading the server config which is rather easy to implement with a cron job. Make sure to test that this actually works before relying on it though.

Thus if you really want to provide forward secrecy you should disable session tickets using ssl_session_tickets off for Nginx and SSLOpenSSLConfCmd Options -SessionTicket for Apache.

Choosing the right cipher suites

Mozilla’s guide on server side TLS provides a great list of modern cipher suites that needs to be put in your web server’s configuration. The combinations below are unfortunately supported by only modern browsers, for broader client support you might want to consider using the “intermediate” list.

ECDHE-RSA-AES128-GCM-SHA256:   \
ECDHE-ECDSA-AES128-GCM-SHA256: \
ECDHE-RSA-AES256-GCM-SHA384:   \
ECDHE-ECDSA-AES256-GCM-SHA384: \
DHE-RSA-AES128-GCM-SHA256:     \
DHE-DSS-AES128-GCM-SHA256:     \
[...]
!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK

All these cipher suites start with (EC)DHE which means they only support ephemeral Diffie-Hellman key exchanges for forward secrecy. The last line discards non-authenticated key exchanges, null-encryption (cleartext), legacy weak ciphers marked exportable by US law, weak ciphers (3)DES and RC4, weak MD5 signatures, and pre-shared keys.

Note: To ensure that the order of cipher suites is respected you need to set ssl_prefer_server_ciphers on for Nginx or SSLHonorCipherOrder on for Apache.

HTTP Strict Transport Security (HSTS)

Now that your server is configured to accept TLS connections you still want to support HTTP connections on port 80 to redirect old links and folks typing example.com in the URL bar to your shiny new HTTPS site.

At this point however a Man-In-The-Middle (or Woman-In-The-Middle) attack can easily intercept and modify traffic to deliver a forged HTTP version of your site to a visitor. The poor visitor might never know because they did not realize you offer TLS connections now.

To ensure your users are secured when visiting your site the next time you want to send a HSTS header to enforce strict transport security. By sending this header the browser will not try to establish a HTTP connection next time but directly connect to your website via TLS.

Strict-Transport-Security:
  max-age=15768000; includeSubDomains; preload

Sending these headers over a HTTPS connection (they will be ignored via HTTP) lets the browser remember that this domain wants strict transport security for the next six months (~15768000 seconds). The includeSubDomains token enforces TLS connections for every subdomain of your domain and the non-standard preload token will be required for the next section.

HSTS Preload List

If after deploying TLS the very first connection of a visitor is genuine we are fine. Your server will send the HSTS header over TLS and the visitor’s browser remembers to use TLS in the future. The very first connection and every connection after the HSTS header expires however are still vulnerable to a {M,W}ITM attack.

To prevent this Firefox and Chrome share a HSTS Preload List that basically includes HSTS headers for all sites that would send that header when visiting anyway. So before connecting to a host Firefox and Chrome check whether that domain is in the list and if so would not even try using an insecure HTTP connection.

Including your page in that list is easy, just submit your domain using the HSTS Preload List submission form. Your HSTS header must be set up correctly and contain the includeSubDomains and preload tokens to be accepted.

OCSP Stapling

OCSP - using an external server provided by the CA to check whether the certificate given by the server was revoked - might sound like a great idea at first. On the second thought it actually sounds rather terrible. First, the CA providing the OCSP server suddenly has to be able to handle a lot of requests: every client opening a connection to your server will want to know whether your certificate was revoked before talking to you.

Second, the browser contacting a CA and passing the certificate is an easy way to monitor a user’s browsing behavior. If all CAs worked together they probably could come up with a nice data set of TLS sites that people visit, when and in what order (not that I know of any plans they actually wanted to do that).

Let the server do the work for your visitors

OCSP Stapling is a TLS extension that enables the server to query its certificate’s revocation status at regular intervals in the background and send an OCSP response with the TLS handshake. The stapled response itself cannot be faked as it needs to be signed with the CA’s private key. Enabling OCSP stapling thus improves performance and privacy for your visitors immediately.

You need to create a certificate file that contains your CA’s root certificate prepended by any intermediate certificates that might be in your CA’s chain. StartSSL has an intermediate certificate for Class 1 (the free tier) - make sure to use the one having the SHA-256 signature. Pass the file to Nginx using the ssl_trusted_certificate directive and to Apache using the SSLCACertificateFile directive.

OCSP Must Staple

OCSP however is unfortunately not a silver bullet. If a browser does not know in advance it will receive a stapled response then the attacker might as well redirect HTTPS traffic to their server and block any traffic to the OCSP server (in which case browsers soft-fail). Adam Langley explains all possible attack vectors in great detail.

One solution might be the proposed OCSP Must Staple Extension. This would add another field to the certificate issued by the CA that says a server must provide a stapled OCSP response. The problem here is that the proposal expired and in practice it would take years for CAs to support that.

Another solution would be to implement a header similar to HSTS, that lets the browser remember to require a stapled OCSP response when connecting next time. This however has the same problems on first connection just like HSTS, and we might have to maintain a “OCSP-Must-Staple Preload List”. As of today there is unfortunately no immediate solution in sight.

HTTP Public Key Pinning (HPKP)

Even with all those security checks when receiving the server’s certificate we would still be completely out of luck in case your CA’s private key is compromised or your CA simply fucks up. We can prevent these kinds of attacks with an HTTP extension called Public Key Pinning.

Key pinning is a trust-on-first-use (TOFU) mechanism. The first time a browser connects to a host it lacks the the information necessary to perform “pin validation” so it will not be able to detect and thwart a {M,W}ITM attack. This feature only allows detection of these kinds of attacks after the first connection.

Generating a HPKP header

Creating an HPKP header is easy, all you need to do is to compute the base64-encoded “SPKI fingerprint” of your server’s certificate. An SPKI fingerprint is the output of applying SHA-256 to the public key information contained in your certificate.

openssl req -inform pem -pubkey -noout < example.com.csr |
  openssl pkey -pubin -outform der |
  openssl dgst -sha256 -binary |
  base64

The result of running the above command can be directly used as the pin-sha256 values for the Public-Key-Pins header as shown below:

Public-Key-Pins:
  pin-sha256="GRAH5Ex+kB4cCQi5gMU82urf+6kEgbVtzfCSkw55AGk=";
  pin-sha256="lERGk61FITjzyKHcJ89xpc6aDwtRkOPAU0jdnUqzW2s=";
  max-age=15768000; includeSubDomains

Upon receiving this header the browser knows that it has to store the pins given by the header and discard any certificates whose SPKI fingerprints do not match for the next six months (max-age=15768000). We specified the includeSubDomains token so the browser will verify pins when connecting to any subdomain.

Include the pin of a backup key

It is considered good practice to include at least a second pin, the SPKI fingerprint of a backup RSA key that you can generate exactly as the original one:

openssl req -new -newkey rsa:4096 -nodes -sha256 \
  -keyout example.com.backup.key -out example.com.backup.csr

In case your private key is compromised you might need to revoke your current certificate and request the CA to issue a new one. The old pin however would still be stored in browsers for six months which means they would not be able to connect to your site. By sending two pin-sha256 values the browser will later accept a TLS connection when any of the stored fingerprints match the given certificate.

Known attacks

In the past years (and especially the last year) a few attacks on SSL/TLS were published. Some of those attacks can be worked around on the protocol or crypto library level so that you basically do not have to worry as long as your web server is up to date and the visitor is using a modern browser. A few attacks however need to be thwarted by configuring your server properly.

BEAST (Browser Exploit Against SSL/TLS)

BEAST is an attack that only affects TLSv1.0. Exploiting this vulnerability is possible but rather difficult. You can either disable TLSv1.0 completely - which is certainly the preferred solution although you might neglect folks with old browsers on old operating systems - or you can just not worry. All major browsers have implemented workarounds so that it should not be an issue anymore in practice.

BREACH (Browser Reconnaissance and Exfiltration via Adaptive Compression of Hypertext)

BREACH is a security exploit against HTTPS when using HTTP compression. BREACH is based on CRIME but unlike CRIME - which can be successfully defended by turning off TLS compression (which is the default for Nginx and Apache nowadays) - BREACH can only be prevented by turning off HTTP compression. Another method to mitigate this would be to use cross-site request forgery (CSRF) protection or disable HTTP compression selectively based on headers sent by the application.

POODLE (Padding Oracle On Downgraded Legacy Encryption)

POODLE is yet another padding oracle attack on TLS. Luckily it only affects the predecessor of TLS which is SSLv3. The only solution when deploying a new server is to just disable SSLv3 completely. Fortunately, we already excluded SSLv3 in our list of preferred ciphers previously. Firefox 34 will ship with SSLv3 disabled by default, Chrome and others will hopefully follow soon.

Further reading

Thanks for reading and I am really glad you made it that far! I hope this post did not discourage you from deploying TLS - after all getting your setup right is the most important thing. And it certainly is better to to know what you are getting yourselves into than leaving your visitors unprotected.

If you want to read even more about setting up TLS, the Mozilla Wiki page on Server-Side TLS has more information and proposed web server configurations.

Thanks a lot to Frederik Braun for taking the time to proof-read this post and helping to clarify a few things!

October 27, 2014 06:00 PM

October 22, 2014

Gavin Sharp (gavin)

r=gfritzsche

I’m happy to announce that Georg Fritzsche is now officially a Firefox reviewer.

Georg has been contributing to Firefox for a while now – his contributions started with done some great work on Firefox’s Telemetry system, as well as investigating stability issues in plugins and Firefox itself. He played a crucial role in building the telemetry experiments system, and more recently has become familiar with a few key parts of the Firefox front-end code, including our click to play plugin UI and the upcoming “FHR self support” feature. He’s a thorough reviewer with lots of experience with Mozilla code, so don’t hesitate to ask Georg for review if you’re patching Firefox!

Thanks Georg!

October 22, 2014 10:14 PM

September 13, 2014

Tim Taubert (ttaubert)

Talk: Keeping secrets with JavaScript - An Introduction to the WebCrypto API

With the web slowly maturing as a platform the demand for cryptography in the browser has risen, especially in a post-Snowden era. Many of us have heard about the upcoming Web Cryptography API but at the time of writing there seem to be no good introductions available. We will take a look at the proposed W3C spec and its current state of implementation.

Slides

Code

https://github.com/ttaubert/secret-notes

September 13, 2014 02:00 PM

August 01, 2014

Margaret Leibovic (margaret)

Firefox for Android: Search Experiments

Search is a large part of mobile browser usage, so we (the Firefox for Android team) decided to experiment with ways to improve our search experience for users. As an initial goal, we decided to look into how we can make search faster. To explore this space, we’re about to enable two new features in Nightly: a search activity and a home screen widget.

Android allows apps to register to handle an “assist” intent, which is triggered by the swipe-up gesture on Nexus devices. We decided to hook into this intent to launch a quick, lightweight search experience for Firefox for Android users.

image image

Right now we’re using Yahoo! to power search suggestions and results, but we have patches in the works to let users choose their own search engine. Tapping on results will launch users back into their normal Firefox for Android experience.

We also created a simple home screen widget to help users quickly launch this search activity even if they’re not using a Nexus device. As a bonus, this widget also lets users quickly open a new tab in Firefox for Android.

image image

We are still in the early phases of design and development, so be prepared to see changes as we iterate to improve this search experience. We have a few telemetry probes in place to let us gather data on how people are using these new search features, but we’d also love to hear your feedback!

You can find links to relevant bugs on our project wiki page. As always, discussion about Firefox for Android development happens on the mobile-firefox-dev mailing list and in #mobile on IRC. And we’re always looking for new contributors if you’d like to get involved!

Special shout-out to our awesome intern Eric for leading the initial search activity development, as well as Wes for implementing the home screen widget.

August 01, 2014 09:10 PM

July 24, 2014

Marco Bonardo (mak)

Unified Complete coming to Firefox 34

The awesomebar in Firefox Desktop has been so far driven by two autocomplete searches implemented by the Places component:

  1. history: managing switch-to-tab, adaptive and browsing history, bookmarks, keywords and tags
  2. urlinline: managing autoFill results

Moving on, we plan to improve the awesomebar contents making them even more awesome and personal, but the current architecture complicates things.

Some of the possible improvements suggested include:

When working on these changes we don't want to spend time fighting with outdated architecture choices:

Due to these reasons, we decided to merge the existing components into a single new component called UnifiedComplete (toolkit/components/places/UnifiedComplete.js), that will take care of both autoFill and popup results. While the component has been rewritten from scratch, we were able to re-use most of the old logic that was well tested and appreciated. We were also able to retain all of the unit tests, that have been also rewritten, making them use a single harness (you can find them in toolkit/components/places/tests/unifiedcomplete/).

So, the actual question is: which differences should I expect from this change?

The component is currently disabled, but I will shortly push a patch to flip the pref that enables it. The preference to control whether to use new or old components is browser.urlbar.unifiedcomplete, you can already set it to true into your current Nightly build to enable it.

This also means old components will shortly be deprecated and won't be maintained anymore. That won't happen until we are completely satisfied with the new component, but you should start looking at the new one if you use autocomplete in your project. Regardless we'll add console warnings at least 2 versions before complete removal.

If you notice anything wrong with the new awesomebar behavior please file a bug in Toolkit/Places and make it block Bug UnifiedComplete so we are notified of it and can improve the handling before we reach the first Release.

July 24, 2014 04:23 PM

May 28, 2014

Marco Bonardo (mak)

Bookmarks backups respin

Part of the performance improvements we planned for Places, the history, bookmarking and tagging subsystem of Firefox, involved changes to the way we generate bookmarks backups.

As you may know, Firefox stores almost everyday a backup of your bookmarks into the profile folder, as a JSON file. The process, so far, had various issues:

  1. it was completely synchronous, I/O on main-thread is evil
  2. it was also doing a lot more I/O than needed
  3. it was using expensive live-updating Places results, instead of a static snapshot
  4. backups take up quite some space in the profile folder
  5. We were creating useless duplicate backups
  6. We were slowing down Firefox shutdown
  7. the code was old and ugly

The first step was to reorganize the code and APIs, Raymond Lee and Andres Hernandez took care of most of this part. Most of the existing code was converted to use the new async tools (like Task, Promises and Sqlite.jsm) and old synchronous APIs were deprecated with loud console warnings.

The second step was dedicated to some user-facing improvements. We must ensure the user can revert to the best possible status, but it was basically impossibile to distinguish backups created before a corruption from the ones created after it, so we added the size and bookmarks count to each backup. They are now visible in the Library / Restore menu.

Once we had a complete async API exposed and all of the internal consumers were converted to it, we could start rewriting the internals. We replaced the expensive Places result with a direct async SQL query reading the whole bookmarks tree at once. Raymond started working on this, I completed and landed it and, a few days ago, Asaf Romano further improved this new API. It is much faster than before (initial measurements have shown a 10x improvement) and it's also off the main-thread.

Along the process I also removed the backups-on-shutdown code, in favor of an idle-only behavior. Before this change we were trying to backup on an idle of 15 minutes, if we could not find a long enough interval we were enforcing a backup on shutdown. This means, in some cases, we were delaying the browser shutdown by seconds. Currently we look for an 8 minutes idle, if after 3 days we could not find a long enough interval, we cut the idle interval to 4 minutes (note: we started from a larger time and just recently tweaked it based on telemetry data).

At that point we had an async backups system, a little bit more user-friendly and doing less I/O. But we still had some issues to resolve.

First, we added an md5 hash to each backup, representing its contents, so we can avoid replacing a still valid backup, thus providing more meaningful backups to the user and reducing I/O considerably.

Then the only remaining piece of work was to reduce the footprint of backups in the profile folder, both for space and I/O reasons. Luckily we have an awesome community! Althaf Hameez, a community member, volunteered to help us completing this work. Bug 818587 landed recently, providing lz4-like compression to the backups: automatic backups are compressed and have .jsonlz4 extension, while manual backups are still plain-text, to allow sharing them easily with third party software or previous versions.

Finally, I want to note that we are now re-using most of the changed code also for bookmarks.html files. While these are no more our main exporting format, we still support them for default bookmarks import and bookmarks exchange with third party services or other browsers. So we obtained the same nice perf improvements for them.

Apart from some minor regressions, that are currently being worked on by Althaf himself, who kindly accepted to help us further, we are at the end of this long trip. If you want to read some more gory details about the path that brought us here, you can sneak into the dependency tree of bug 818399. If you find any bugs related to bookmark backups, please file a bug in Toolkit / Places.

May 28, 2014 05:11 PM

May 09, 2014

Stephen Horlander (shorlander)

(Re)Designing Firefox

Update — 2014/05/28

I added a slideshow of some of the design iterations we went through from Firefox 4 to Firefox 29.

Warning: It’s image heavy, might take a while to load on a slow connection.

So, last Monday we launched a thing. You may have noticed it. We called it Australis. Now it’s just called Firefox.

We spent a lot of time on it.

Dedicating yourself to a project can become an intense experience. You think about it all the time: in the morning, at dinner, when you are trying to watch a movie, in the shower, in your sleep, when you should be sleeping…

But then you set it free, because it’s finally mature enough for that. It’s exciting. But it’s also really scary. You are never sure if you made the right decisions along the way. I like to take some time in the post release glow for some reflection.


History

Left: Firefox 4, Right: Firefox 29

So how did we get from there to here?

We launched Firefox 4 in March 2011. It was a big change from Firefox 3. It introduced the Firefox button, revamped the add-ons manager, removed the status bar, combined the stop, go and reload buttons and included a comprehensive visual update—all while still having time to prototype and discard some other features along the way.

And yet it wasn’t perfect. It had a lot of the rough edges that projects accumulate in the process of going from being designed and built to being shipped.

Firefox 4 was our last monolithic release before we moved to a rapid release cycle. Six week cycles seemed like the perfect timeframe to iteratively smooth out the rough edges. So I created a project to do just that.


Philosophy

The project that I had created for iterative refinement however quickly transformed into a significant overhaul.

People in the frame: Sinchan Banerjee, Alex Faaborg, Brian Dils, Trond, Stephen Horlander (my legs), Jennifer Boriss, Frank Yan, Alex Limi
Taking the picture: Madhava Enros

At the beginning of June the UX team met up for its first post Firefox 4 team offsite. On the agenda was figuring out “What’s next?”. The entire team gathered in a room to pitch ideas and talk about problems unresolved—or that had been introduced—during the development of Firefox 4.

One theme that had been floating around for a while rose to the surface— Firefox is about customization, it should feel like it’s yours.

What would this mean for the interface we had just shipped? A lot of ideas were tossed around. Eventually one guiding principal stuck—make the best core experience we can and allow users to add and change the things that matter the most to them.

Building a fun easy to use Customization Mode—along with a more flexible Firefox Menu—would become the foundation of the new Firefox.


Visual Profile

So Curve, Such Aerodynamic

The offsite also sparked a set of other ideas that would make up what became known as Australis. Primarily: unifying the disparate bookmarking elements in the main window, refining the visual design, consolidating related or redundant features and streamlining the tabstrip.

While the redesigned customization mode would be core to the experience—the redesigned tabstrip would change the entire profile of Firefox.

“Make it look faster. No, no! It needs more air vents!”
Firefox 4 Aerodynamic Sketch — 2010

We had explored the idea of adding visual cues to Firefox to make it feel faster and smoother before. Yet some of the ideas were a little over the top.

Curvy Tab Sketch — June, 2011

This sketch from the design session—inspired by a previous mockup from Trond—had a curvy tab shape that immediately resonated with everyone.

It also had one important additional design tweak—only render the tab shape for the active tab. Highlighting the active tab reduces visual noise and makes it easier to keep your place in the tabstrip.

The early curve shape tried on a few looks. At first it was too angular, then it was too curvy, then it was too short, then it was too tall and then (finally) it was just right.

Design ideas for background tabs on Windows — October 2011

It turns out that designing background tabs without a tab shape is a lot easier if you have a stable background to work with. Windows 7 has translucent windows of variable tints and Windows 8 has flat windows of variable color. This meant we needed to create our own stable background.

We went through several variations to ensure that the background tabs would be legible. First we tried creating a unified background block, but it seemed too heavy. We even thought about keeping background tab shapes and highlighting the active tab in some other way.

Eventually we decided on a background “fog” that would sit behind the tabs and in front of the window. Think of it as an interface sandwich—glass back, curvy-tab front with a delicious foggy center.

We also made sure that adding curves didn’t increase the width of the tabs taking up precious tabstrip real estate. And we removed the blank favicon placeholder for sites without favicons. A small tweak that frees up some extra room for the title.


Menu Panel and Customization

Redesigning the Firefox Menu

One size really doesn’t fit all with something as complex and personal as your web browser. Add-ons have always made Firefox a thoroughly customizable browser, however arranging things has never been a great experience out of the box.

But before we could tackle the customization behavior, we had to rethink the Firefox Menu.

The main toolbar is the primary surface for frequently used actions; while the menu is the secondary surface for stuff you only use some of the time. We wanted users to be able to move things between these surfaces so they could tailor Firefox to their needs.

For the updated layout we started with the idea of a visual icon grid. This grid nicely mapped to the icons already found on the toolbar and would ideally make moving things back and forth feel cohesive.

Larger 3×3 Grid with Labels—by Alex Faaborg

The first grid we designed was wider, with smaller icons and no labels. This quickly changed to a three column grid with larger labeled icons. The updated layout served the same goal but was more clear and also less cramped.

Not everything we currently had in the Firefox Menu would translate directly to the new layout. We had to add special conjoined widgets—also known as “Wide Widgets”—for Cut/Copy/Paste and the Page Zoom controls. We also added a footer for persistent items that can’t be customized away. We did this to prevent users from getting into a broken layout with no escape hatch.

We had some serious debates about whether to use a an icon grid or a traditional menu list. The visual grid has some drawbacks: it isn’t as easy to scan and doesn’t scale as well with a lot of items. But the icon grid won this round because it was more visual, more inline with what we wanted out of drag-and-drop customization and had the side benefit of being touch friendly.

Some items from the Firefox Menu had submenus that could’t be easily reduced to a single action: Developer Tools, History, Bookmarks, Help and Character Encoding. Nested submenus hanging off of a panel felt pretty awkward. We had several ideas on how to handle this: inline expando-tray™, a drawer(interactive mockup—click History) and in-panel navigation.

We settled on what we call subviews(interactive mockup—click History). Subviews are partial overlays containing lists that slide in over the menu. Click anywhere in the partially visible menu layer to get back to the main menu—or close the entire menu and reopen it.

With the new menu layout Firefox should hopefully work just fine for most people out of the box. But by using the new Customization Mode you can really tailor Firefox to your needs. If you are interested in knowing more about why what is where Zhenshuo Fang wrote a great post about it.


Customize All the Things!

Now we needed to figure out how update Firefox’s aging customization experience. Things started off a little ambitious with the idea that we could combine toolbar customization, themes and add-ons into the same UI. This led to an early interactive demo (it does some stuff, hint: tools icon –> plus sign) to try and see if this would work.

This demo surfaced a few issues: 1) including add-ons was going to be complicated 2) we needed a direct representation of the menu panel instead of a proxy. This led to a bunch of mockups for a flatter interface sans add-ons integration. Eventually I made a second demo without the theme selector that is closer to what we ended up shipping. Then Blake Winton turned that into an awesome prototype that actually did something.

Final Customization Mode

The demos and prototypes helped us quickly get feedback from people on the idea. One of the complaints we heard was that it wasn’t obvious that you were entering a mode. We mocked up a lot of ideas for various ways to give visual feedback that you were in a mode and could now rearrange your stuff. We eventually settled on a subtle blueprint pattern and Madhava suggested we add some kind of animation with padding.


Wrapping it up

Thank you to everyone who dedicated so much time and effort into making this happen.

If you want to know more about the people and process behind Firefox 29, Madhava has a good post with an overview.

I think the post-release glow is over now. Time to get back to making Firefox better.

May 09, 2014 03:43 PM

May 05, 2014

Christian Legnitto (LegNeato)

Firefox 29′s best feature

Apparently Firefox 29 had some major feature or something. I don’t know much about that. What I do know about is a new Firefox feature that will finally cause Firefox to beat Chrome once and for all. Yes ladies and gentlemen, I am talking about a revolutionary feature I’ve dubbed:

Scheme typo fixer-upper™©

Have you ever copy and pasted URLs into the location bar and seen this screen instead of the page you wanted?

notfound

Of course you have. So you double check the URL and go “ooooooooh, I missed the h in http when copying and pasting.” Stupid me. Well, at least Firefox gave me an obtuse message to tell me what is going on. How lovely.

So, this happened to me one too many times. And I got pissed. And I had a conversation with my browser. I was probably drinking so it made sense at the time. The conversation went something like this:

WTF Firefox. You KNOW what I meant. Stop being a dick and just do the right thing™.

And then it reminded me of one of the very first “cool” things I experienced at Facebook. Facebook uses Phabricator and Arcanist for their development workflow. Arc (like Mozilla’s Mach) is a local tool to run builds, send up diffs, manage patches, and things of that persuasion. One of the things you do with arc is run `arc build`.

Well guess what happened. My first day at Facebook I typed `arc biuld` instead of `arc build`. I’m such an idiot. They should have fired me. Much to my delight I saw this:

arc

It says:

Assuming 'biuld' is the British spelling of 'build'...

And it hit me. Someone was THINKING. They encountered this problem, thought about how many people would do that typo, realized there was really only one possible intended result, and decided to have the computer just do the right thing™. Someone cared about engineer time so much they fixed it for every future engineer to come. The thought and whimsical message made such a large impression on me I insisted we add a similar feature to Buck, Facebook’s super-fast Android build tool:

$ ./bin/buck biuld
 No sign of buck.jar -- building Buck!

It uses Levenshtein distance to figure out what you really meant to type. Humans: 1, machines: 0. Sweet.

So right about here I am likely supposed to point out Facebook (and specifically my team) is hiring. Cool, I’ve now done my duty as a manager–that should keep the recruiting attack dogs at bay.

Where was I? Oh yeah, man over machine. So I started thinking, which is never a good thing. This sort of thing happens to me every day and slows me down a lot. For example, if I type `git cone` instead of `git clone`:

cone

It says:

$ /usr/bin/git cone
git: 'cone' is not a git command. See 'git --help'.

Did you mean this?
    clone

You know DAMN WELL what I meant git, and you mock me by echoing it out right in front of me. You spit in my face and just sit there all smug about it. What a dick.

So I went to teach git some manners, and it turns out the feature is already there. If you set help.autocorrect git will do the right thing™. There was a bug about making it default but I lost interest reading it to figure out why they insist on making humans do the work of computers. As an aside, do you know the git developers only take contributions as patches to a mailing list? Coding like we’re in the 90s, whooooooo!

Get. To. The. Point.

What does this have to do with Firefox you might ask? What, you don’t like my stories? Fine! Remember the screenshot at the beginning:

notfound
I started thinking how stupid Firefox was being, how many wasted human-hours were spent reading the message and acting on it, and how much goodwill Firefox loses when its users roll their eyes at this screen.

And it made me angry. So I decided to fix it. How hard could it be? Alcohol may have been a factor in my difficulty calculation. But I did it! The proof is a screenshot of Google:

elgoog

I typed in “ttps://google.com” and was taken to “https://google.com” automatically without seeing an error screen! Firefox will now fix all these typos:

  ttp://  → http://
  ttps:// → https://
  tps://  → https://
  ps://   → https://
  ile://  → file://
  le://   → file://

Humans: 2, machines: 0.

Err, ok, this feature doesn’t screenshot well. Just trust me it works. But you probably don’t trust me. I wouldn’t. Try it yourself in any flavor of Firefox > 29! Go ahead, I’ll wait.

Did it work? Of course it did! Oh damn, you tried “hptts” and it didn’t get fixed? My change only deals with dropped characters and not jumbled character typos. It’s not a bug because I documented it™. To be honest I am lazy and lost interest after I fixed my copy and paste pain but it would be trivial to extend my patch to cover the latter case. Get in touch and I can mentor you! Take a look at the code (part 1, part 2). And there are tests! I mainly wrote those to not be a hypocrite as I preach the unit test gospel to product engineers every day. Oh, and for “quality”.

Turns out this life-changing feature was a lot more involved than expected, for two reasons.

First, I naively assumed there would be one location where everything gets loaded from the location bar input. To me that was a sensible way of organizing things. I could then just do a string match on the URL scheme and replace it with the right stuff…easy peasy, right? Errr, no. Turns out Firefox is…interesting. And on a lot of platforms with different UI code.

I kept digging deeper in the stack and when I got to where I needed (docshell) it wasn’t the right place for the fix at all. I’d potentially break web compatibility. Whoops. Rather than doing a simple fix at a lower-level chokepoint I needed to patch in optional support to that low-level and then hunt down every place Firefox takes in URLs and do the typo fixup. And then of course my first patch didn’t work due to a last-minute change and I had to fix things up in a second patch. See the gory details in the bug.

Second, Mozilla’s current setup is very developer-hostile. And that isn’t just “OMGWTFBBQ why isn’t Firefox on teh GitHubz!” and “lke dis if u cry evry tme u tuch c++”. Seriously, I do not know how Firefox developers get any work done. I would quit if I was forced to work on Firefox code all day. I’m not talking about code quality, I’m talking about supporting stuff–from build times, to no linters, to crufty code-search tools, to horrible code review, to dealing with no CI. I think I’ve been spoiled at Facebook as we invest heavily in speeding up the development workflow (we’re hiring!). I have more thoughts on this I intend to blog later, but to my Mozilla friends it doesn’t need to be like this. Expect more from your tools and development workflow!

Ahem. I’ll get off my soapbox and save my fire and brimstone tools talk for another day.

El fin.

Please enjoy the best feature Mozilla has shipped since Firefox 1.0 – the Scheme typo fixer-upper™©! If you encounter problems feel free to file bugs on me and I will promptly ignore them.

May 05, 2014 02:10 PM

April 30, 2014

Margaret Leibovic (margaret)

Firefox Hub Add-on Hackathon

For the past few months, the Firefox for Android team has been working on Firefox Hub, a project to make your home page more customizable and extensible. At its core, this feature is a set of new APIs that allows add-ons to add new content to the Firefox for Android home page.

These APIs are new in Firefox 30, but there are even more features available in Firefox 31, which is moving to Aurora this week. As we’ve been working on these APIs, we’ve been building plenty of demo add-ons ourselves, but we’re at the point where we want more developers to get involved!

Next week (May 5-9), we’re holding a distributed add-on hackathon to encourage more people to start building things with these APIs, and I want to encourage anyone who’s interested to participate!

This hackathon has three main goals:

To kick off this hackathon, I made an etherpad that links to documentation, example add-ons, and a long list of new add-on ideas. Since our community lives all over the world, we’ll hang out in #mobile on irc.mozilla.org to ask questions, report problems, and share progress on things we’re making. In addition to building add-ons, you can also participate by testing new add-ons as they’re made!

At the end of the week, everyone who participated in the hackathon will receive a special limited-edition open badge, as well as pride in contributing to Firefox for Android. And maybe I’ll try to dig up some special prizes for anyone who makes a really cool add-on :)

April 30, 2014 05:40 PM

April 29, 2014

Matthew Noorenberghe (MattN)

Thanks for reviewing screenshots of Firefox's new look!

Thanks to the 150 participants in just over 24 hours who reviewed screenshots of the new Firefox look which was released today! This was an amazing success for the first time using the online screenshot review tool and led to around 1000 pieces of feedback given. I've now summarized all of the feedback given (over 90 unique issues) and have begun filing bugs or associating reports with existing issues on file. The top contributors were syssgx, MarcN, db48x, Vykintas, & zombie. Send me a message to claim your prize. Given the success, we'll likely be using the tool again for UI changes in the future so watch for participation opportunities in the coming months. You can download the new version of Firefox now or check for updates from the about dialog. Thanks!

April 29, 2014 09:50 AM

April 15, 2014

Matthew Noorenberghe (MattN)

An easy way to test the New Firefox Beta look and feel before it's released

Screenshot of the new Firefox UI on Windows 7 with the menu panel openThe new Firefox Beta is faster, simplified and easier to customize and we need your help to test it out before it gets released in a few weeks time. There is now an easy way to review Firefox user interface changes without even installing the new version. "How is that possible?" you might ask. We have a collection of hundreds of screenshots of the new Firefox in various different configurations (affecting features such as tabs, toolbars, themes, customization mode, and the new menu) that are ready for you to review. It's fast and easy to do in three simple steps:
  1. Open up the screenshot review tool and enter a nickname to log in (there may be a small reward for the most valuable contributions so keep that in mind when choosing)
  2. A random screenshot will be displayed where you can simply identify any visual issues related to the new user interface that you may see. Simply drag to select the region of the image and add a comment (and optionally a bug number). See an example.
  3. When you're done reviewing that image, simply click the button to get another and go to step 2. Endless fun ensues!
Of course, installing Firefox Beta, testing the functionality and filing bugs is still really valuable and encouraged. Areas to focus on include the new customization mode, menu panel, tabs, and Firefox Account Sync. So, what are you waiting for? Start reviewing now.

April 15, 2014 01:14 AM

April 07, 2014

Tim Taubert (ttaubert)

A ready-to-use virtual build environment for Firefox

If you ever wondered what contributing to Firefox feels like but you never had the time to read and follow through our instructions to setup a build environment or wanted to avoid screwing around with your precious system then this might be for you.

This article will guide you through a small list of steps that in the end will leave you with a virtual machine ready to modify and build your own development version of Firefox.

I hope this will be valuable to novice programmers that do not have a full C++ development environment at hand as well as to the more experienced folks with little time and lots of curiosity.

Install VirtualBox

Note: The Open Virtualization Format (OVF) is supported by other Virtualization Environments such as VMWare, etc. You can use those if already installed instead of VirtualBox.

Go to the VirtualBox Downloads page and download the latest version available for your operating system. Should you already have VirtualBox installed then please ensure you are running the latest version by checking for updates before continuing.

Download the Firefox Build Environment

Now is the time to download the virtual machine containing our development environment ready to modify and build Firefox. You can get it here:

http://vmimages.mozilla.net/ovf/FirefoxBuildEnv.ova
(sha1 = 2bc7ea3d831163e507129353b0c5306ab30f18aa)

Downloading ~2.8 GB might take a while if you are on a slow connection, sorry.

Set up the virtual machine

Once the image has been downloaded you can double-click the .ova file and import the new virtual machine into VirtualBox. Please give it at least 2048MB of RAM (4096MB if you can) and the same number of processors that your host machine has available. Building Firefox takes up a lot of resources and you want it to build as fast as possible.

Now that your virtual machine is ready, boot it and wait for the Ubuntu desktop to be shown. A terminal will pop up automatically and do some last steps before we can get started. After a successful installation Sublime 2 should start automatically.

Note: Should you ever need root credentials, use “firefox-dev” as the password. If you want to change your Language and Keyboard settings then follow the instructions on How to change the UI Language in Ubuntu.

Build Firefox

Click Tools > Build to start the process. This might take a long time depending on the features of your host machine, please be patient. You can watch the build progress in the text editor’s console at the bottom. Once the build has finished you can use Tools > Run to start your custom Firefox build and check that everything works as expected.

Note: if you want to switch from an optimized to a debug build then choose Tools > Build System > Firefox (Debug) and hit Tools > Build again to start a debug build.

Now what?

You successfully built Firefox for the first time and wonder what’s next? How about picking a small bug for a start, contribute code and get your changes shipped to half a billion people? If that sounds compelling then take a look at Bugs Ahoy! and find something to work on that sounds interesting to you.

If you are interested in digging deeper into the build system or the version control system, or want to know more about how to create your first patch and post it to our bug tracker then take a look at our Code Firefox Lessons.

I would love to hear your feedback about the Firefox Build Environment! Please tell me what can be improved and what you would like to see in the next version. Do not hesitate to drop me a mail should you have a more detailed opinion.

April 07, 2014 04:00 PM

April 04, 2014

Tim Taubert (ttaubert)

Starting my fourth year at Mozilla

Today marks the beginning of my fourth year at Mozilla. It has been an amazing three years and the best job I could hope for. Since March I am now in the position of an Engineering Manager with a few highly intelligent and great people that I am very grateful to call my team.

I am super excited about all the personal and professional challenges I will be facing this year. It is my core belief that it is all about growth and for that Mozilla is exactly the right place to be.

<3

April 04, 2014 09:00 AM

March 27, 2014

Gavin Sharp (gavin)

even more new firefox reviewers

I’m pleased to announce two additions to the list of Firefox reviewers:

Please join me in congratulating Mike and Florian!

March 27, 2014 03:00 AM

March 23, 2014

Matthew Noorenberghe (MattN)

PSA: Deployed Firefox profiles should not contain profile lock files

In short

If you are making a base Firefox profile which will be deployed to other machines, be sure to delete the profile lock files before creating the template/image. The filenames to delete vary by operating system and can be found in the roaming profile directory: When a lock file exists, the modification timestamp of that file needs to reflect the time when that profile was locked/used and so having a lock file from months ago when a base profile or image is made will lead to incorrect results.

Longer version

In the past (prior to bug 294260 in Firefox 13), the profile lock files (mentioned above) would be deleted upon successful shutdown of the Mozilla application. Since bug 294260, we now leave the profile lock files in place and simply update the existing file (and thus its modification time). We use the modification time and a preference to detect startup crashes related to the profile. In this case that means a crash from the time of locking the profile to the browser window appearing for 30 seconds. If the timestamp of the profile lock file doesn't match what was stored in the preferences, we assume that there was a crash in the last startup before it got to updating the preference. After three consecutive startup crashes detected within 6 hours, we will ask the user if they want to reset their profile or enter safe mode to resolve the crashes.

Offering to reset unused profiles

More recently, we added a new use of the last time of profile use in order to detect profiles which hadn't been used in a while (60 days) and offer to reset them (bug 498181 in Firefox 25). The idea is that there are people returning to Firefox after having been away for a while using another browser and they'd like to give Firefox another try. If they left Firefox due to technical issues such as slow startup or crashes, we don't want them to return to that problem and so we offer a reset as a way to get a fresher Firefox experience while still preserving key data such as bookmarks, history, passwords, etc. (the list is outlined in the reset dialog). If the profile lock file is part of a base profile or image like DeepFreeze and its timestamp is more than 60 days ago users may get prompted to reset immediately. Some users have already run into this and that's what prompted this email. In summary, the profile lock file must accurately reflect when the profile was last locked. Deleting it from base profiles is the best way to do this. Let me know in the comments if you have any questions.

March 23, 2014 09:00 PM

March 18, 2014

Brian Bondy (bbondy)

Does Modern UI Firefox usage indicate Windows 8.1 modern UI is in trouble?

On Friday, Mozilla announced it would not ship its Modern UI Firefox browser due to low adoption.

Is Windows 8.1 modern UI in trouble?

No.


Did Mozilla make the right decision with Metro given the current circumstances?

Yes.


Modern UI Firefox usage, in Mozilla's measurements, is not necessarily a true reflection of Modern UI usage in general.

Modern UI Firefox Usage != Modern UI usage

I do believe that Microsoft's modern UI is important for touch hardware, and I do believe that touch hardware is something people are adopting and will adopt more.


I believe that the Modern UI Firefox usage was low, at least in part, for 2 specific reasons:

  1. Microsoft doesn't allow your browser to run in Modern UI unless you are the default browser. Several people could have had a Modern UI capable Firefox pre-releases installed, but just never knew it.

  2. Microsoft makes it a lot harder to set your browser as the default in Windows 8. Before Windows 8, each browser could prompt you, and then they could set your default for you. As of Windows 8 you need to ask first, then tell Microsoft to show a prompt that shows a list of browsers (confusing). And that only sets the HTTP default. If you want all defaults, such as HTML and HTTP, then you have to send the user to control panel, make them search for the browser, then make them select your browser and set all defaults.


It would be great if Microsoft could fix these issues around default status. More competition leads to better software, and having good software on your platform is important. Every Windows Modern UI user loses when there's only one Modern UI browser choice.

March 18, 2014 07:38 PM

March 14, 2014

Brian Bondy (bbondy)

Switching careers, next steps

I made one of the hardest decisions of my life, I'll be leaving Mozilla, a company that I thought I'd be at forever. Mozilla has amazing people and culture, and is easily the best job I've ever had.

I plan to remain a contributor, both on the Code Firefox site, and on Firefox Desktop itself.


I'll be joining Khan Academy, a like minded, mission oriented, open and non-profit company.

Khan Academy provides a free, world class education, to anyone, anywhere.

I'm leaving because the Khan Academy mission speaks more to me personally.

Along with that mission comes a lot of thoughts I believe in, such as:

I'll be joining the Computer Science department at Khan Academy in particular. I'm extremely excited for this new opportunity and I can't wait to do amazing things at Khan Academy.

March 14, 2014 08:58 PM

March 06, 2014

Dave Townsend (Mossop)

Developer Tools meet-up in Portland

Two weeks ago the developer tools teams and a few others met in the Portland office for a very successful week of discussions and hacking. The first day was about setting the stage for the week and working out what everyone was going to work on. Dave Camp kicked us off with a review of the last six months in developer tools and talked about what is going to be important for us to focus on in 2014. We then had a little more in-depth information from each of the teams. After lunch a set of lightning talks went over some projects and ideas that people had been working on recently.

After that everyone got started prototyping new ideas, hacking on features and fixing bugs. The amount of work that happens at these meet-ups is always mind-blowing and this week was no exception, even one of our contributors got in on the action. Here is a list of the things that the team demoed on Friday:

This only covers the work demoed on Friday, a whole lot more went on during the week as a big reason for doing these meet-ups is so that groups can split off to have important discussions. We had Darrin Henein on hand to help out with UX designs for some of the tools and Kyle Huey joined us for a couple of days to help work out the final kinks in the plan for debugging workers. Lot’s of work went on to iron out some of the kinks in the new add-on SDK widgets for Australis, there were discussions about memory and performance tools as well as some talk about how to simplify child processes for Firefox OS and electrolysis.

Of course there was also ample time in the evenings for the teams to socialise. One of the downsides of being a globally distributed team is that getting to know one another and building close working relationships can be difficult over electronic forms of communication so we find that it’s very important to all come together in one place to meet face to face. We’re all looking forward to doing it again in about six months time.

March 06, 2014 01:52 AM

March 05, 2014

Dave Townsend (Mossop)

An editable box model view in the devtools

This week the whole devtools group has been sequestered in Mozilla’s Portland office having one of our regular meet-ups. As always it’s been a fantastically productive week with lots of demos to show for it. I’ll be writing a longer write-up later but I wanted to post about what I played with over the week.

My wife does the odd bit of web development on the side. For a long time she was a loyal Firebug user and a while ago I asked her what she thought of Firefox’s built in devtools. She quickly pointed out a couple of features that Firebug had that Firefox did not. As I was leaving for this week I mentioned I’d be with the devtools group and she asked whether her features had been fixed yet. It turns out that colour swatches had been but the box model still wasn’t editable. So I figured I could earn myself some brownie points by hacking on that this week.

The goal here is to be able to inspect an element on the page, pull up the box model and be able to quickly play with the margins, borders and padding to tweak the positioning until it looks right. Then armed with the right values you can go update your stylesheets. It saves a lot of trial and error with positioning.

It turned out to be relatively simple to implement a pretty full version. The feature allows you to click one of the box model values and type whatever value you like, in any CSS unit you prefer. If the size had been set in the stylesheet in some specific unit then that is what appears in the input box for you to change. Better yet as you type numbers the element updates in the page on the fly and you can use the arrow keys to increase/decrease the value until you’re happy. It’s a really natural way to play with the element’s position.

The changes made appear on the element so you can find them in the rule view pretty easily. This patch is based on an updated version of the box model view which is why it looks so different to existing Firefox, all my work does is make the numbers editable.

I actually completed this so quickly that I decided to take this one step further. One thing missing from the box model display is information about border colours. So I added some colour swatches for each border and made them editable with the regular devtools colour picker.

Both of these patches are pretty much complete but they’ll have to wait for the new box model highlighter to be complete before they can be reviewed and land.

March 05, 2014 05:29 PM

Brian Bondy (bbondy)

Investing in our community through Code Firefox

Mozilla 2014 Goal

As Deb Cohen (VP, Chief of People) said:
Our goal this year is to find new ways for people to become part of us. This year we have a very aggressive goal, for all of us to look inside our teams, our work, our organization, and create some of these pathways.


What is Code Firefox?

The Code Firefox site (@codefirefox), is a new video and exercise framework which is highly inspired by Khan Academy. Its purpose is for on-boarding contributors and sharpening the skills of existing community members.

People learn differently. Providing different ways to consume information is crucial to growing Mozilla. For many people, Code Firefox provides an easier, and more consumable approach to learning how to contribute to Mozilla.

Would you like to start contributing patches to Mozilla? Start here


Testimonial

One prime example of Code Firefox's usefulness is of a new contributor who joined last month, and actively uses Code Firefox. He has already submitted 18 patches.

In a personal email Anuj Agarwal (Student at Vellore Institute of Technology) writes:
Just for fun, here is the patch I got reviewed today! And it was made possible only because of your tutorials. Thank you!


Code Firefox statistics

The site has not been promoted yet, but it already has:


What has Code Firefox accomplished?

51 videos, 3 exercises focused mostly on on-boarding new Firefox developers
Videos are captioned
Ability to track watched videos and completed exercises when logged in
Exercise framework that parses code and gives immediate feedback and experience

Videos and Exercises are accessible and searchable through tags


Where is the Code Firefox site going? (Code Firefox Goals)

Opening up the ability to post videos to other people
Ability to create different "Outline tracks", for example, a QA track.
More video and exercise content being added, initially for Firefox OS and Desktop
Integrate with Mozilla Open Badges
Working more closely with Student Ambassadors
Implement a point system
Interoperability with Mozillians in creative ways
Implementing hints for exercises

Other goals


Highlights of benefits gained by Mozilla from the site

Brings in new contributors
Retains new and existing contributors
Sharpens skills of existing members of our community
Code Firefox uses and promotes Persona
Code Firefox will use and promotes Open Badges

Will train employees with advanced skills with advanced tracks


Media coverage of Code Firefox so far

Some people have taken the time to write about Code Firefox:


What our community is saying


Would you like to create Code Firefox videos?

Please contact me at bbondy[at]gmail.com if you'd like to make a series of videos on Code Firefox for your area of expertise.

March 05, 2014 03:49 PM

March 03, 2014

Margaret Leibovic (margaret)

Dare 2B Digital 2014: Remix the Web!

Dare 2B Digital is an annual South Bay conference that brings 300 young women ages 12-16 together to encourage them to consider STEM fields in college by coming together for a full day of inspiring talks and workshops showcasing women’s work and relevance in technology. For the past four conferences, Lukas has signed Mozilla up as a sponsor and created a workshop that is run 3 times that day and reaches about 80-100 attendees. Last year, Lukas and I created a workshop to teach these girls how to create mobile HTML5 games. This year, Lukas couldn’t make it to the conference because she was busy organizing another great outreach event, so I volunteered to organize our workshop, and I recruited Katie and Heather to help me.

I really love Webmaker, a Mozilla project dedicated to helping people learn how to create amazing things on the web, so I decided to create a workshop to teach the girls how to use some of the Webmaker tools. My goal was to teach the girls how to use these tools to understand the basic building blocks of the web, as well as show them how easy it is to make their own web content. To prepare for the conference, I gathered 20 loaner laptops, ordered some swag, and remixed an existing Webmaker teaching kit to create an outline for the workshop.

We started off each workshop with a brief overview of how the web works (Katie used her teacher skills to create some nice whiteboard diagrams), and then we jumped right into using the Webmaker tools. We started by walking the girls through using the X-Ray Goggles to inspect webpages, and taught them that they can modify what they see in their browsers. We definitely heard lots of “This is so cool!!!” when we showed them how to change text and images on any webpage.

Once they felt comfortable with X-Ray Goggles, we pointed them to a gallery of existing Webmaker projects, and showed them how to use Thimble to remix those projects into their own. At this point, we let them loose to start working on their own remixes, and there was lots of enthusiasm and laughter as they started working on their own projects. We also felt thoroughly out of touch with youth culture when they started filling their projects with images of young male celebrities we had never even heard of.

Unfortunately, we only had an hour and 15 minutes for each workshop, which flew by, but we encouraged the girls to make their own Webmaker accounts to continue working on their remixes at home. They were definitely really excited to learn that all of these resources are available to them at home, and that they only need a web browser to get started!

March 03, 2014 07:15 PM

February 14, 2014

Johnathan Nightingale (johnath)

Four

FourFour is a big deal. Big girl bed. Big girl bike. You explain this to me, in case I’d missed it.

“Soon I will be four, and then I’ll be a teenager.”

Eef.

I travel a lot these days. Not constantly, probably not even 25% of the time, but enough that I often go a week without giving you a hug. And being the overthinker dad that I am, that makes me wonder what effect it has on you. We video chat when I’m gone, whenever the internet connection is good enough. When it isn’t, I record videos for you and send them to your mom to play. I’ve never missed a weekend with you, though it’s sometimes been quite a trick to make that work. I think I’m a really present dad, I try to be, but I wonder how you feel about it, and how you’ll think about it years from now. I hope you’ll agree.

My dad, your grandpa, tells me that when I was a kid and he had a weekend with me, he felt protective. Like he didn’t want to share. Like he wanted me all to himself. And how that wasn’t very helpful and we had a much better time once he got past it and made room. Boy do I understand that, now. Time with you is the best thing, and sometimes I don’t want to share. But grandpa’s wise, and sharing you with others is great, and watching you venture off and be social and own a room is just wonderful. But I’m still glad that you let me read you stories and tuck you into bed at night.

Today was your birthday and you ran around like an idiot with a birthday cake tiara and a birthday bear and made goofy faces any time I tried to take your picture. You ordered pizza and oreo cake and tried to splash me with bathwater. And you fell asleep while I stroked your hair and you snored a little and it was the best thing ever.

Goodnight, Lil. Sweet dreams. I love you.

February 14, 2014 03:03 AM

February 04, 2014

Margaret Leibovic (margaret)

WIP: Home Page Customization in Firefox for Android

In Firefox 26, we released a completely revamped version of the Firefox for Android Home screen, creating a centralized place to find all of your stuff. While this is certainly awesome, we’ve been working to make this new home screen customizable and extensible. Our goal is to give users control of what they see on their home screen, including interesting new content provided by add-ons. For the past two months, we’ve been making steady progress laying the ground work for this feature, but last week the team got together in San Francisco to bring all the pieces together for the first time.

Firefox for Android has a native Java UI that’s partially driven by JavaScript logic behind the scenes. To allow JavaScript add-ons to make their own home page panels, we came up with two sets of APIs for storing and displaying data:

image

During the first half of our hack week, we agreed on a working first version of these APIs, and we hooked up our native Java UI to HomeProvider data stored from JS. After that, we started to dig into the bugs necessary to flesh out a first version of this feature.

image

Many of these patches are still waiting to land, so unfortunately there’s nothing to show in Nightly yet. But stay tuned, exciting things are coming soon!

February 04, 2014 03:45 AM

January 09, 2014

Felipe Gomes (felipc)

Hello 2014

Happy new year everyone! During the holidays I had been thinking about my goals for 2014, and I thought it’d be a good exercise to remember what I’ve done in the past year, the projects I worked on, etc. It’s nice to look back and see what was accomplished last year and think about how to improve it. So here’s my personal list; it doesn’t include everything but it includes the things I found most important:

Related to performance/removing main-thread-IO:

Related to hotfixes:

Related to electrolysis:

Other things:

Events attended:

And:

That’s it. Looking forward to a great 2014!


January 09, 2014 07:09 PM

December 03, 2013

Matthew Noorenberghe (MattN)

Introducing a Google spreadsheet library to fetch Talos results

As part of the performance investigation for Australis (the in-progress Firefox desktop redesign), we wanted to be able to easily track Talos performance benchmark numbers across all of the relevant platforms without having to open 52 graph server links (Datazilla wasn't ready at the time) . My goal was to use the graph server API to pull in the data and display the regression percentage for all of the tests we were tracking in an overview. Rather than writing a new tool from scratch, I decided to look into Google Apps Script, which I had only heard about before, and it seemed to be able to help implement what I wanted on top of Google Spreadsheets. The result is a shared Talos Google Apps Script Library [Google login required] (revision log) (API docs) that anyone can use. You can see it in use for TART and other tests for Australis along with the demo spreadsheet.

How to use the Talos library in a Google spreadsheet

Also see Google's documentation on Libraries and the demo spreadsheet.

  1. In your Google Spreadsheet, choose Tools > Script Editor
  2. In the new tab, click Close to skip the tutorials and templates.
  3. In the menu, choose Resources > Manage Libraries. If you are asked to save your project, do so.
  4. In the "Included Libraries" window, paste the following in the "Find a Library" textbox and click Select: MCurDezZ1B01NQX34URNDye9_pJh_2yq6
  5. The "Talos" library should have appeared above. Choose a version of the library (the latest is probably best), and click Save.
  6. Replace the contents of the default script file ("Code.gs" at the time of writing) with pass-through wrappers for every function you want to use (example). Let me know if there is a better way to do this as it's not ideal.
  7. Call your wrapper functions from within your spreadsheet. e.g. =getTalosResult(226, 59, 25, "ca7577238ef4"). You can get test, branch and platform IDs from the compare.py script or from the URLs output on TBPL.
Now you can use the power of spreadsheets to slice and dice the data as you please. Perhaps you like custom graphs?

Caveats

Eventually the library should switch to using Datazilla but this works for now since talos results are being reported to both services. If you would like to contribute changes to the library, let me know.

December 03, 2013 10:22 AM

December 02, 2013

Johnathan Nightingale (johnath)

NSID 2013 – Travelogue

To everything there is a season.

I’m driving today. The road races by, cold with the oncoming winter. The trees, skeletal without their leaves, look dead. The fields are frozen, shorn of their fruits long ago. Picturesque. But even from within the heat of the car, the sight of it all makes me shiver.

The seasons make us. They mete out our minutes of sunlight and degrees of warmth and millimeters of rain. Not with a parent’s loving tenderness, or even a teacher’s careful pedagogy, but with a metronome’s mindless repetition. And today they have made the world outside this car cold, and still, and dead.

But you are not bound by their rhythm! You are vibrant, and alive, and your own self. What the seasons decree must be a cold and barren time you can make into growth and flourishing life. Hooray for you, that have this freedom and this power to make your world better than it was before!

To shave is to suppress. So much suppression infuses what we do that, hoodwinked by fashion, we become our own gaolers. We shave ourselves. And as the seasons swirl around us to make us cold, to make us withdraw, in that moment, friends, I say that we should fight back against their mindless cruelty.

One month of the year I will not go gently into my cell. One month of the year I will make, upon my person, rebellion against desolation around me. One month, this month, I will be as beautiful as I am able. I will take no blade against myself, I will not lessen me. And I invite you. I entreat you. From the cold of this frozen highway I beg you to do the very same.

And then tweet about it, and post pictures on flickr.

noshavingindecember.org

December 02, 2013 02:40 AM

November 18, 2013

Brian Bondy (bbondy)

Short one video per concept Firefox development videos

I'd like to introduce a new site I'm working on named Code Firefox.


Elevator pitch

Becoming a Mozilla contributor can be intimidating. There's a lot of information to consume, and that information can be dispersed in a variety of different places. This site aim to outline the process of becoming a Mozilla contributor, from the start, with gentle, short, consumable 2-10 minute videos.


A long time coming

In July 2012 I blogged about my first year at Mozilla, and where I'd like to go in the future.

I'm starting work on a project to bring videos for code contributors. The plan is to have short 5-10 minute videos that each cover a single topic. This would be similar to Khan Academy videos in the 1 video per concept way, but will be more like a tutorial and will be focused on contributing to Mozilla.

It took me a while to actually start on that goal, partly because I kept making excuses for why it wasn't a good time to start yet.

Maybe I was just scared of presenting something in a way that wasn't the best way. But when it comes down to it, I know a lot of people can benefit from the content of the videos I'm creating. And because of that, I'm no longer afraid to be corrected.

A couple month ago, sitting in the airport coming home from the Mozilla 2013 Summit, I registered codefirefox.com. I also created a github project and started coding the site that would be used for those videos.


The future of Code Firefox

Code Firefox aims to focus on Desktop Firefox videos from the ground up. I'd love to also do videos on Firefox for Android, Firefox for Windows 8 style UI, and especially for Firefox OS.

I'd like to eventually get to a place where the videos can be used for onramping new hires, and perhaps also used by some schools as the basis of a class.


Status and focus

Since I began posting videos, I've been posting an average of over 1 video per day, I'm close to 20 videos so far.

These videos are not focused on perfection, partly because I'm far from it, and partly because I have so much content to go out, that there is more of a benefit to do new videos instead of focusing on old ones.

One statistic I found interesting is that the average visit duration on my blog is under 1 minute; whereas, the average visit duration of the Code Firefox site is over 5 minutes.


The process

I’m using ScreenFlow on OS X to make the videos in a Windows 8 VMware Fusion VM, and a Blue Yeti microphone.

As each video is posted, I post a message about it on this twitter account.

November 18, 2013 03:52 AM

November 06, 2013

Brian Bondy (bbondy)

Shared profiles for Metro Firefox and Desktop Firefox

Up until recently, Metro Firefox was designed to use a separate profile from desktop Firefox. We had planned to rely on sync to bring the data from the 2 profiles together.

We've had feedback around this, and we've found that a better experience would be obtained by sharing a single profile.

We're in the process of implementing shared profiles.


What is a profile?

A profile in Firefox stores all of your personal settings, including: bookmarks, preferences, extensions, passwords, history, and more.

We'll be sharing profiles from both applications, but there are some exceptions:


What are the upsides of having a shared profile?

Sharing a single profile would allow you to instantly retain bookmarks, login status, cookies, and other profile data between environments.

You would never need to wait for some kind of 'sync service' to complete its work to see your data in the other environment.

Another task highly related to sharing a single profile is adding a "Relaunch in Desktop" option from Metro Firefox and a "Relaunch in Metro" option from Desktop Firefox.


Are there any downsides to having a shared profile?

Yes, but we believe them to be corner case issues that the user can work around, see below.

The main disadvantage is that before you could run both Metro and Desktop browsers at the same time. This will change so that you can only run either one or the other at any given time.

Windows 8 is a clash of 2 environments:

  1. a Desktop environment which is optimized for mouse and keyboard
  2. an immersive environment which is optimized for touch

If you're using a keyboard and mouse, and since we will be having a shared profile, there is no reason you'd want to use the Metro browser. If you really want to you can, but surely you don't need to use both Desktop and Metro at the same time.

The opposite argument is also true for touch. Most of the time when using touch you'll want to use the Metro browser, but there are special cases where you'll want to switch, like if you need an addon or plugin to take effect.

For users who really want to run both at the same time, they can force the use of a different profile using the -ProfileManager command line argument from Desktop Firefox.


Will having a shared profile change the release date?

Having a shared profile is not expected to change the release of Metro Firefox. A smooth async panning and zooming experience is being worked on before we want to release, and during that time we're implementing shared profiles.

The switch to Desktop / Metro feature may not retain all opened tabs and filled in form status in v1 if time does not permit. We plan to use session restore to do this, but there's some non trivial work to do there.


Where is this work being tracked?

Shared profile work is being tracked in bug 924860.


Other implementation details


Current progress

If you'd like to see general progress of Metro Firefox you should use either the Nightly or Aurora builds.

Current progress for profile sharing can be seen in Oak builds (Not recommended for general use). These patches will also move to mozilla-central (Nightly builds) after QA'ing is done to ensure there is no harm to the Desktop profile.

The completed list of tasks related to this feature so far include:


Feedback

Constructive feedback is appreciated and wanted. Please feel free to leave a comment here or post in the profile sharing tracking bug: Bug 924860.

November 06, 2013 02:46 PM

October 29, 2013

Margaret Leibovic (margaret)

MozFest Recap: Building Your First Mobile Web App

This past weekend I attended Mozilla Festival in London, and it was incredibly energizing! Everyone at MozFest was so excited about the web, but unlike your average tech conference, there was a much more diverse set of participants, including hackers, makers, journalists, educators, and even lots of kids. The focus of this year’s MozFest was about creating a web-literate world, so I decided to facilitate a session on building your first mobile web app.

Although projects like Webmaker and Appmaker are designed to make content creation more accessible for everyone, I wanted to create some learning materials that could put people on the path towards becoming more serious web developers. This path includes learning the fundamentals of HTML/CSS/JS, as well as how to use the developer tools built into browsers, and popular collaboration tools like Github.

To teach these basics, I made a slide deck that walks you through building a simple stopwatch app that’s hosted using Github pages. At the start of my session, I presented these slides (and of course threw in some Firefox dev tools demos!), but then I let everyone loose to start hacking at their own speeds. There were about 25 participants with a wide range of ability levels, and I think this format worked well to keep everyone engaged (and gave me the opportunity to offer more individual help).

My hope is that other people might find these resources useful, so please use them and share them! Also, because they’re open source, pull requests are welcome :)

October 29, 2013 08:27 PM

October 15, 2013

Margaret Leibovic (margaret)

Remote Developer Tools and Firefox for Android

A lot of the Firefox for Android UI is written in Java, but under the hood there’s still quite a bit of JS that controls core browser logic. We also use HTML to create the content of most of our chrome-privileged pages, such as about:addons. Today I found myself working on a polish bug for about:apps. In the past, tweaking the CSS for these pages has been a pain, since testing changes required pushing a new APK to your device. However, I remembered that the remote inspector recently landed for Firefox 26, so I decided to try it out. Verdict: it was amazing!

imageimage

Setting up the remote developer tools is as easy as flipping prefs on desktop and mobile (these prefs are now exposed in the UI!), running an adb command to set up port forwarding, and hitting a button. The directions are really well documented on MDN, so you should look there for more detailed instructions.

While playing around with the inspector, I also realized that remote chrome debugging is enabled for Firefox for Android. It wasn’t immediately obvious to me that in order to attach the debugger to the main chrome window, you need to select the “Main Process” link on the desktop “Connect to remote device page”, but once I did that, I had access to all of the browser chrome JS. And since add-ons for Firefox for Android are written in JS, this means that you can easily debug all of your add-on code!

image

Big thanks to the Firefox Developer Tools Team for doing so much awesome work. And I know they’re always looking for new contributors to help them out!

October 15, 2013 04:42 AM

October 01, 2013

Felipe Gomes (felipc)

GSoC 2013 completed

Last week, the 2013 edition of the Google Summer of Code officially finished. The Firefox team had come up with a few project ideas for this year and I mentored Abhishek Choudhary in the project dubbed “about:memory for real people”. The goal was to produce a friendlier and prettier version of about:memory that could be used by users and web developers to investigate the memory usage aspects of their webpages.

Mentoring GSoC was a nice experience again (it was my 4th time as a mentor) and I’m happy to say that the project was completed successfully! On Sep 16th, the “soft pencils down” goal on GSoC timeline, Abhishek tagged the 1.0 version on GitHub and the code was called complete. After a small extra bug fix, 1.0.1 was tagged and then submitted to addons.mozilla.org!

The project itself was very interesting because it was not a straightforward coding exercise: there was also a creative element to it which allowed us to experiment different visualization styles to try and see what worked best. It began with a few different separate visualizations but it evolved over time to one main integrated viz that supports seeing the data consumed by each tab and to watch how it evolves over time. The user can take manual time snapshots or let it take one every 5 seconds, and then use the time slider to go back and forth between the last 5 snapshots. The main hope is that this should be useful for a web developer who wants to see how a particular action on the website might affect its memory usage and if it fits the expectations (does that operation increase memory? does it release it as expected?).

Here’s a screenshot of the main interface:

fx-statistics screenshot

You can see more details and install it from AMO. There’s a small catch: the add-on currently won’t work with Aurora or Nightly, just Beta and Release. This was an interesting part of the project: on the few last weeks of the project, the API that the add-on had been using changed, and it was not possible to change it to the new API among the other wrap-up tasks for the project. But luckily add-on versioning works well, and as the goal of this add-on is end users, shipping for beta and release is ok for now, and we’ll soon be posting a new version that is compatible with FF25+.

Abhishek also has a very nice blog post that you should be reading instead of mine :), describing the project progression and his experience working on it. I want to take this opportunity to say congrats to him and thanks to all students who participated in the GSoC project this year! I hope everyone had a nice time and hopefully the experience have interested some of you to continue to be part of community! Looking forward to next year’s projects.

 


October 01, 2013 04:37 PM

September 27, 2013

Margaret Leibovic (margaret)

Home Page Snippets for Firefox for Android

Have you ever noticed the messages that appear under the search box on desktop Firefox’s home page? We call these messages snippets, and they’re dynamically created from web content hosted by Mozilla. These snippets provide an easy way to communicate with desktop users, but right now there’s no equivalent feature for Firefox for Android.

Since we first released the native version of Firefox for Android, we’ve included a hard-coded promotional banner as part of our home page — you’re probably seen it encourage you to set up Firefox Sync. However, we’ve been working on a complete rewrite of the home page for Firefox 26 (exciting!), and we’re taking the opportunity to replace this with a new customizable banner.

Because this banner is part of our native Java UI, we can’t just load web content the way desktop does. Instead, we decided to build a JS API to let add-ons customize it. We came up with a simple API that lets developers add and remove items from a set of messages that rotates through this banner. Right now, these messages just consist of an icon, some text, and a click handler. We put this banner API in a new Home.jsm module, since we have plans to add more home page customization APIs to this module in the future!

As a demo, I made an add-on that fetches data from a basic snippets server, and turns that data into messages that are added to the home banner. This server currently serves data from the 5 most recent @FennecNightly tweets, which is actually pretty relevant to Nightly users. If you’re using Nightly, I’d encourage you to try it out, and as always, please file any bugs you find!

September 27, 2013 01:05 AM

September 17, 2013

Margaret Leibovic (margaret)

Reader Mode Improvements in Firefox 24

Firefox 24 was released today! Along with a lot of other great features, this release includes a bunch of improvements to Reader Mode. The most noticeable improvements are a polished options menu and a “Night Mode” that automatically changes the background/text colors depending on ambient light levels. However, many other small bugs were fixed in the process, increasing the quality of this feature.

I’m making this post because I want to highlight that over half of the bugs fixed for these Reader Mode improvements were fixed by volunteers! I want to thank the 7 volunteers who fixed these 12 Reader Mode bugs. You all rock!

September 17, 2013 09:10 PM

September 03, 2013

Justin Dolske (dolske)

Mozilla office history

If you don’t live in the San Francisco Bay area, you may not be aware that the culmination of a major infrastructure project is underway this holiday weekend. The Bay Bridge, one of 5 major Bay-area bridges, is in the middle of a 5 1/2 day closure as it’s transitioned over to its replacement. (The other 4 major bridges, in case you’re wondering, are the
Richmond–San Rafael Bridge, the San Mateo Bridge, the Dumbarton Bridge, aaaand… hmmm… oh, right, the Golden Gate bridge)

The Bay Bridge was originally built in the 1930s, and after the Loma Prieta earthquake in 1989 it became clear it needed to be replaced. One of the flashbulb memories many people have of the quake — in addition to it interrupting the ’89 World Series and the collapse of the double-decker Cypress Street Viaduct — is the failure of a span on the Bay Bridge, with cars driving over it. Since then, the western half of the Bay Bridge has been retrofitted to be earthquake-safe, but the eastern span of the bridge has taken longer to completely replace. This weekend’s work is to transition the connection points, so that tomorrow people will be driving over a completely new bridge that’s been 11 years and $6.4 billion in the making!

(I’m getting to the part where Mozilla ties into this.)

Last Friday @BurritoJustice tweeted a link to a slideshow that dove into the engineering history of the Bay Bridge, complete with photos taken during the construction.

It’s some fantastic engineering porn, and I spent my lunch reading through all of it. I happened to notice that the building in the background of one of the photos looked familiar…

Mozilla’s San Francisco office, in the historic Hills Brothers Coffee building at 2 Harrison Street, is literally next-door to where the western span of the Bay Bridge lands in S.F. It makes for some really great views of the bridge from our top-floor patio:

As well as a first-row seat beneath the giant “HILLS BROS COFFEE” sign atop the building.

It’s this sign that made it easy to spot our building in the engineering history slideshow. The building was constructed in 1926, and the Bay Bridge wasn’t built until 1933-1936, so I was curious to see if the sign was visible in other contemporaneous photos. I started digging though some online resources, and got lucky right away by finding a highres version of that picture at the Library Of Congress:

I skimmed through the other 415 photos in this collection and another 1160 from UC Berkeley (So You Don’t Have To™), and found some other nice shots with the Hills Coffee sign peeking out from the background:






So there you have it. Pics of the Mozilla San Francisco office from both ends of an 80 year span of history.

September 03, 2013 01:58 AM

August 21, 2013

Justin Dolske (dolske)

On Firefox’s Password Manager

It’s been a while since I last blogged about Firefox’s password manager. No big deal, it really hasn’t fundamentally changed since since I rewrote it 6 years ago for Firefox 3.0. Well, ok, the storage backend did switch to SQLite, but that’s mostly invisible to users. There’s a noteworthy change coming soon (see next post!), but I figured that would be easier to explain if I first talked about how things currently work. Which I’ve been meaning to do for a long time, ahem.

The first thing you should know is that there is no standard or specification for how login pages work! The browser isn’t involved with the login process, other than to do generic things like loading pages and running javascript. So we basically have to guess about what’s going on in order to fill or save a username/password, and sometimes sites can do things that break this guesswork. I’m actually surprised I don’t get questions about this more often.

So let’s talk about how two of the main functions work — filling in known logins, and saving new logins.

Filling in a known login

The overall process for filling in an existing stored login is simple and brutish.

  1. Use the chrome docloader service and nsIWebProgress to learn when we start loading a new page.
  2. Add a DOMContentLoaded event listener to learn when that page has mostly loaded.
  3. When that event fires…
    1. Check to see if there are any logins stored for this site. If not, we’re done.
    2. Loop through each form element on the page…
      1. Is there an <input type=password> in the form? If not, skip form.
      2. Check to see if any known logins match the particular conditions of this form. If not, skip form.
      3. Check to see if various other conditions prevent us from using the login in this form.
      4. Fill in the username and/or password. Great success!

Phew! But it’s the details of looking at a form where things get complex.

To start with, where do the username and password go? The password is fairly obvious, because we can look for the password-specific input type. (What if there’s more than one? Then we ignore everything after the first.) There’s no unique “username” type, instead we just look for the first regular input field before the password field. At least, that was before HTML5 added more input types. Now we also allow types that could be plausibly be usernames, like <input type=email> (but not types like <input type=color>). Note that this all relies on the order of fields in the DOM — we can’t detect cases where a username is intended to go after the password (thankfully I’ve never seen anyone do this), or cases where other text inputs are inserted between the actual username field and the password (perhaps with a table or CSS to adjust visual ordering).

And then there’s the quirks of the fields themselves. If your username is “alice”, what should happen if the username field already has “bob” filled in? (We don’t change it or fill the password.) Or, more common and depressing, what if the username field already contains “Enter your sign in name here”? In Mongolian? (We treat it like “bob”.) What if the page has <input maxlength=8> but your username is “billvonweiterheimerschmidt”? (We avoid filling anything the user couldn’t have typed.)

And then there’s the quirks of the saved logins. What if the username field already has “ALICE” instead of “alice”? (We ignore case when filling, it’s a little trickier when saving.) Is there a difference between <input name=user> and <input name=login>? (Nope, we stopped using the fieldname in Firefox 3 because it was being used inconsistently, even within a site.) What about a site has both a username+password _and_ a separate password-like PIN? (Surprisingly, we were able to make that work! Depending on the presence of a username field, we prefer one login or the other.)

And then. And then and then and then. Like I said, there’s no spec, and sometimes a site’s usage can break the guesses we make.

Saving a new/changed login

In comparison, the process for saving a login is simpler.

  1. Watch for any form submissions (via a chrome earlyformsubmit observer)
  2. Given a form submission, is there a password field in it? If not, we’re done.
  3. Determine the username and password from the form, and compare with existing logins…
    • If username is new, ask if user wants to save this new login
    • If username exists but the password is different, ask if user wants to change the saved password
    • If username and password are already saved, there’s nothing else to do.

Of course, there are still a number of complicating details!

This whole process is initiated by a form submission. If a site doesn’t actually submit a form (via form.submit() or <button type=submit>), but just runs some JavaScript to process the login, the password manager won’t see it. And thus can’t offer to save a new/changed login for the user. (Note that this is easy for a site to work around — do your work in the form’s onsubmit, but return |false| to cancel the submission).

Oh, and there’s still the same question as before — how to determine which fields are the username and password? We reuse the same algorithm as when filling a page, for consistency. But there are a few wrinkles. The form might be changing a password, so there could be up to 3 relevant password fields (for the cases of: just a new password, old and new passwords, and old + new + confirm.). And the password fields could be in any order! (Surprisingly, again, this works.) The most common problem I’ve seen is an account signup page with other form fields between the username and password, such as “choose a user name, enter your shipping address, set a password”. The password manager will guess wrong and think your zipcode is your username.

Oh, and somewhere in all this I should mention how differences in URLs can prevent filling in a login (or result in saving a seemingly-duplicate login). Clearly google.com and yahoo.com logins should be separate. But we also match on protocol, so that a https://site.com login will not be filled in on http://site.com. And what about www.foo.com and foo.com or accounts.foo.com? (We treat them separately.) What about mail.mozilla.com and people.mozilla.com? (Also separate.) What you might not realize is that we also use the form’s action URL (i.e., where the form is submitted to), ever since bug 360493. While this prevented sending your myspace.com password to evilhacker.com, it also breaks things when a site uses slightly different URLs in various login pages, or later changes where their login forms submit to.

Oh, bother.

All the gory details

This is one of the benefits of being Open Source. If you want to see alllll the gory details of how the Firefox password manager works, you can look at the source code. See http://mxr.mozilla.org/mozilla-central/source/toolkit/components/passwordmgr/. In particular, LoginManagerContent.jsm contains the code implementing the stuff discussed in this post, with the main entry points being onContentLoaded() and onFormSubmit().

Finally (!), I’ll mention that the Firefox password manager has some built in logging to help with debugging problems. If you find it not working in some particular case, you can enable the logging and it will often make the problem clear — or at least clearer to a developer when you file a bug!

August 21, 2013 12:54 AM

August 13, 2013

Johnathan Nightingale (johnath)

Three and a Half

Lily eating a pearWell, Lil, I have some news for you: I think you’re an introvert.

I don’t want to box you in, and anyhow it’s more of a continuum than a yes/no, but I see things I recognize. The way you play for an hour by yourself without any outside stimulation. The way you stand at the edge of a group of kids at the park, watching them and trying to figure them out. You come by it honestly, but it still makes my heart hurt a bit. Not because I disapprove; far from it. Introverts are some of my very favourite people, and also pretty in vogue among the techie, cerebral, introspective types I tend to run with. But introversion is tricky. You’ll pick up labels like “shy” and “awkward” and you’ll either take those in as parts of your identity or you’ll rebel against them. Or maybe you won’t. Maybe you’ll be a perfectly well-adjusted human. It is a pretty consistent folly of your dad’s to extrapolate from today to your entire future life. It’s probably silly and I probably shouldn’t, but I do. And sometimes it makes my heart hurt.

Speaking of extrapolation, I’ve been thinking about who, what, and where you’ll be when you read these. I imagine you in your 20s. I don’t know why. I don’t know how I expect to keep them secret from you that long. But I think of you reading them in your 20s, all of them in a sitting, and I wonder what you’ll want to know about being three and a half. You like pears and corn on the cob and sausages with mustard. You love the beach. You’re in circus school on Sunday mornings. You find the word “toots” hilarious. Your youtube favourites are vihart’s math videos, and goats yelling like humans.

That will change. You change quickly now. You went away to California for a few weeks to visit some friends and you came back a different kid. It made me feel protective for a minute. Like I didn’t want you influenced by the outside world. Which is stupid, because those influences are wonderful kids, but also because your life is going to be full of other influences and the last thing I would want to do is stop it. But parenthood is a conservative force – we love you the way you are, and we resist change that we didn’t initiate. Forgive me that, if it happens again when you’re old enough to notice. It’s not a rational response; it’s a reflex. Parental lizard brain.

The world will be different, too. Right now, we — the collective we — worry about some things. We worry about the planet – we think we’re heating it up and we think that’s going to cause big problems. We’re spending a frustrating lot of time debating the evidence and agreeing with each other, but we also have lots of excellent people trying to make it better. We worry about humanity – it’s getting easier for one angry person to hurt a lot of people, and the fear of that is pushing governments into reactions straight out of science fiction: armed drone patrols, global mass surveillance. I don’t talk about this much with the three and a half year old you, but I want the adult you to know that we see these things happening, and we’re not sure how to fix them, but we’ll try. I have hope that we’ll succeed.

Right now, that’s not your biggest concern, though. And that’s fine. Right now your biggest concerns are whether there is ice cream at the beach (there is), and whether tonight is a hair-washing night (it is), and whether I will read you one more story (I will). Thank you for adding so much silly to my life. Thank you for asking “why?” about everything. Thank you for sneaking quietly into the bedroom on Saturday morning, staring at me inches from my face until I startle awake, and then announcing that it’s time for pancakes.

I love you, Lil, and I love the person you’re becoming.

Daddy

 

August 13, 2013 01:27 PM

August 07, 2013

Margaret Leibovic (margaret)

Increasing Volunteer Participation on the Firefox for Android Team

Over the past year, the Firefox for Android team has really seen increased volunteer participation. It’s been amazing to see our community grow, and I want to share some of the strategies we’ve been using to engage volunteers.

1. Make it easy to get started. We maintain a "Get Involved" page with links to a bunch of resources to help newcomers get started, including clear and up-to-date build instructions. We also share some guidelines for how exactly to get started, including things like making sure you have a build environment set up before trying to dive into a bug, and how to reach out to us when you need help.

2. Be available to help. Our team is always on IRC, and we’re quick to answer questions. We also make an effort to respond to Bugzilla comments/requests in a timely fashion, especially if we see it’s someone’s first bug. Helping newcomers can require patience, but being responsive is a really important part of keeping volunteers engaged.

3. Create contribution opportunities. We mark bugs as mentor bugs as often as we can. In addition to simple “good first bugs”, we also try to mark more challenging bugs as mentor opportunities. The key with that is to set clear expectations about how challenging we expect the bug to be, and redirect newcomers to easier bugs if they find themselves in one that’s too hard.

4. Recognize contributors. We made a "Friends of the Mobile Team" section of our weekly meeting notes, where we call out volunteer contributions that were made over the past week. We recently created a badge that we award to anyone who appears in this section (people love badges!) and we also try to publicize important contributions with our @FennecNightly Twitter account.

5. Develop a culture of mentorship. This last point is a bit harder to act on, but it’s the most important. A big part of why we have more (and stronger) volunteer contributors is that we’ve taken the time to mentor them. On our team, most developers have mentored at least one bug (if not many), and make an effort to mark bugs as mentor bugs whenever they can. We also make a big effort to remain patient and friendly, and I think that attitude has spread through our community. One of the greatest rewards of developing this culture is seeing new contributors evolve into mentors themselves. It is an amazing feeling to watch some of the volunteers I mentored helping newcomers write their first patches, and do so with so much kindness and patience.

I’ve been having lots of fun bringing new people into our team, and I’d love to help other teams at Mozilla do the same. Please reach out to me if you have any questions, or share any other tips I may have omitted in the comments below!

August 07, 2013 12:25 AM

July 18, 2013

Gavin Sharp (gavin)

changes to the list of Firefox reviewers

I’ve made a few updates to the list of Firefox reviewers:

Please join me in welcoming the new reviewers – send patch review requests their way! :)

July 18, 2013 10:26 PM

July 17, 2013

Gavin Sharp (gavin)

measuring Firefox responsiveness improvements with jydoop

One of the other cool things that came out of the Firefox Desktop work week that I mentioned in my previous post was increased exposure to a handy tool written by taras and bsmedberg: jydoop. Benjamin has a post explaining what it is and how it works, but I wanted to describe one specific use: measuring main-thread responsiveness issues in the wild.

First, a bit of backstory!

One of my team’s overarching goals for a long time has been eliminating all sources of main-thread file I/O.[1] One of the worst main-thread I/O offenders in Firefox code is our SQLite Storage API. A few years ago we went a bit overboard using a version of this API that is synchronous, and does its file I/O work on the main thread. So recently we’ve been focused a lot on eliminating all users of that API.

One consumer of this API that we fixed in Firefox 23 is the Form History code (responsible for remembering information entered into web forms and offering to re-fill it). Neil Deakin, markh and mak worked together to fix the core service (based on earlier work from a previous Mozilla intern, Felix), but after landing that improvement, taras noticed that there were still some remaining issues. We eventually determined that the likely primary culprit for the remaining Form History main-thread I/O was Firefox Sync’s use of the older Form History API. markh and rnewman worked together to fix Sync, but we weren’t entirely sure whether that was sufficient to eliminate all of the main thread I/O we were seeing reported by telemetry.

Now, to the jydoop part

Taras had identified that our initial Form History fix wasn’t complete using the SlowSQL telemetry dashboard. This gives a high-level overview of slow SQL statements occurring in the wild, and he noticed that after we landed the original fix for form history, some formhistory.sqlite statements were still showing up. Unfortunately, the SlowSQL dashboard only offers two time ranges: previous week, and the previous month, and it aggregates data across all builds from a specific channel (e.g. “nightly”), which means that I couldn’t easily use it to confirm whether our Sync fix was effective.

That’s where jydoop came into the picture. It allows much more sophisticated analysis of the SlowSQL data. I wrote a couple of jydoop scripts: one to summarize the slowSQL reports for formhistory.sqlite from builds before the Sync fix (i.e. Nightly builds with a build ID < 20130623) and another to do the same for builds containing the fix. I then ran those scripts against Telemetry reports from the period from 2013-06-21 to 2013-06-24 (inclusive, i.e. roughly two days on either side of the landing of the fix).

The results!

The results are pretty conclusive: the reports of slow SQL statements for formhistory.sqlite from builds including the Sync fix were dramatically reduced. The most frequently slow statement from builds before the fix (“COMMIT TRANSACTION”) was responsible for a total of 417 seconds of main-thread unresponsiveness (174993 occurences of slowness at an average of 143ms of “jank” at a time), over that period. The Sync fix prevented that statement (and many other slow statements) from occurring at all, thus removing a significant cause of main-thread responsiveness issues experienced by Firefox users in the wild.

Next steps

If you look at the results in more detail, they reveal that there are still some slow SQL main-thread operations happening on formhistory.sqlite – they’re less frequent and severe, and are tracked by other bugs. I’m going to be using jydoop to gather more data on the improvements my team’s already made, as well as gauge the potential for improvement in areas we’re still working on. I also want others on the team to learn to use jydoop, so that people can make their own measurements to validate the improvements they’re working on.

[1] Main-thread file I/O is a term used to describe trying to read from or write to the disk on the same thread (the “main thread”) as you’re trying to respond to user input (e.g. mouse/key events) and do other things like drawing the UI. Reading from or writing to the disk can take a relatively long time, particularly if you have a slow hard drive, so we want to always do it on a background thread so that those other important responsiveness tasks (responding to input and updating the UI) aren’t affected.

July 17, 2013 06:18 PM

high-level introduction to Firefox front-end code

At the end of June the Firefox desktop team held a Performance-themed work week Toronto, which included representatives from the Performance team as well as some other teams (QA, automation/testing, project management, etc.). During the week a bunch of us got together and held a little “intro to Firefox code” session – partially for the benefit of a new hire on our team (Steven MacLeod), but also partially because it can be fun to share different perspectives on how we understand Firefox. This was a group discussion, but a lot of it involved me giving my perspective on how Firefox front-end code works at a high level, and how it all fits together. We also went off on a few tangents discussing Mozilla development in general.

mconley was kind enough to take some notes from the session on an etherpad (UPDATE: Etherpad is having issues, here’s an alternate link). These could serve as a good base for an MDN article, or perhaps something similar to https://wiki.mozilla.org/Gecko:Overview.

July 17, 2013 12:09 AM

July 16, 2013

Felipe Gomes (felipc)

FISL 14

Firefox, Tux and Gnu at FISL

Two weeks ago, almost the entire Mozilla community in Brazil assembled together in Porto Alegre to attend FISL, the largest free software conference in the country, which brings together about 7000 people every year to talk about every side of open source: communities, business, government, education, etc.

I think the community had its best attendance ever this year. For one thing, it was the largest group: there were about 30 contributors from all over the country who contribute to every area of Mozilla. For another, our participation this year was entirely planned and run by the community, and everyone was working together in sync to make it a success.

Our activities included various talks on the main conference track, an apps hackaton, a Mozilla room, SUMO sprints, and just a lot of engagement with the students and developers at the conference. As expected, interest in Firefox OS was very high, and this brought to us a lot of people who wanted to hear more about the OS, take a look at the devices (we had a few Geeksphones for demonstrations!) and learn how to build apps. With the dedicated Mozilla room we were able to run several “Apps 101″ sessions to get people started on apps and the hackaton (at times back-to-back sessions to keep up with the demand!), and to provide a welcoming place for them to hack on the apps or to get some hands-on help when they needed.

Mozilla Room at FISL

Welcome to the Mozilla Room at FISL 14

In true free software spirit, most of the talks at FISL are live streamed and also recorded, and you can see the full program here. Below are all the talks we had (I think I’ve included them all but apologies if I forgot any).

(p.s.: if you want to see an entertaining explanation of how Persona works I highly recommend you watch François’ presentation at the 11’30” mark)

The hackaton ran during the conference and finished on Saturday afternoon. We got a total of 7 apps submissions, which are all posted on github, and the winner was announced at the end of Surman’s talk and received a Geeksphone Keon! There were also e-books as prizes for other teams. Big props to Adriano, André Garzia and Morvana for running the hackaton!

Hackaton Prize

With all that, what I have to say is thanks to all the contributors who helped with all our activities. It was great to meet again some of the old-timers and specially nice to meet the newer members of the community to whom I had only known online before. I was very impressed with everyone’s dedication and passion for Mozilla! Attending FISL is always a special experience for me because that’s where I started my true involvement with open source and Mozilla. After meeting Marcio and Fabricio in São Carlos in 2005 and been following things online for a while, I attended my first FISL in 2008 where I gave a presentation about add-ons development and there I met Chofmann, Blizzard, Taras, Mitchell, Mary, and that was a big push for me to start contributing. I’ve been attending the event ever since, and nowadays I often get people asking my story and how they too can get involved and I hope that I can too serve as support and inspiration for the new people joining who wants to participate and grow in the community. Last but not least, a special thanks to Sérgio for being the lead organizer and volunteering an incredible amount of his time in the past N months to make FISL happen this year!

Until FISL 15!


July 16, 2013 09:42 AM

July 02, 2013

Felipe Gomes (felipc)

Firefox Performance Week

Last week, the Firefox front-end team got together in Toronto to discuss various of our ongoing projects, with a special focus on performance-related goals. The performance team also joined us as there are various joint projects between our teams. After the week is over it’s interesting to look back and see what was accomplished during the week and what it opened as guidance for our future work. Here is my personal recap, some of the highlights of what I did and what I saw during the week. In no particular order:

And beyond that there were also various other topics that benefited from having the team all together in one place, specially Social API and Australis. To me personally the week was immensely helpful as a motivator and as a way to get consensus and clarity into our upcoming projects, not to mention the fine chance of directly interacting with the entire team which not often happens as we’re all spread around the around!

As a side note, this blog post was an exercise in itself as one of the outcomes for me from the Summit Assembly meeeting we had at the week before. There’s more on it that needs a separate blog post to explain, but a personal action that I’ve taken from that event is to increase the transparency and involvement with the community about the work I do.


July 02, 2013 06:35 AM

May 21, 2013

Dave Townsend (Mossop)

Six years revisited

Two years ago I blogged about how it had been six years since I wrote my first patch for Firefox. Today I get to say that it’s been six years since I started getting paid to do what I love, working for Mozilla. In that time I’ve moved to a new continent, found a wife (through Mozilla no less!), progressed from coder to module owner to manager, seen good friends leave for other opportunities and others join and watched (dare I say helped?) Mozilla, and the Firefox team in particular, grow from the small group that I joined in 2007 to the large company that will soon surpass 1000 employees.

One of the things I’ve always appreciated about Mozilla is how flexible they can be about where you work. Recently my wife and I decided to move away from the bay area to be closer to family. It was a hard choice to make as it meant leaving a lot of friends behind but one thing that made it easier was knowing that I wouldn’t have to factor work into that choice. Unlike many other companies I know there is no strict requirement that everyone work from the office. True it is encouraged and it sometimes makes sense to require it for some employees, particularly when starting out, but I knew that when it came time to talk to my manager about it he wouldn’t have a problem with me switching to working remote. Of course six years ago when I started I was living in the UK and remained so for my first two years at Mozilla so he had a pretty good idea that I could handle it, and at least this time I’m only separated from the main office by a short distance and no time-zones.

The web has changed a lot in the last six years. Back then we were working on Firefox 3, the first release to contain the awesomebar and a built-in way to download extensions from AMO. Twitter and Facebook had only been generally available for about a year. The ideas for CSS3 and HTML5 were barely written, let alone implemented. If you had told me back then that you’d be able to play a 3D game in your browser with no additional plugins, or watch videos without flash I’d have probably thought they were crazy pipe-dreams. We weren’t even Jitting our JS code back then. Mozilla, along with other browser makers, are continuing to prove that HTML, CSS and JS are winning combinations that we can build on to make the future of the web open, performant and powerful. I can’t wait to see what things will be like in another six years.

May 21, 2013 12:05 AM

May 15, 2013

Dave Townsend (Mossop)

Firefox now ships with the add-on SDK

It’s been a long ride but we can finally say it. This week Firefox 21 shipped and it includes the add-on SDK modules.

We took all the Jetpack APIs and we shipped them in Firefox!What does this mean? Well for users it means two important things:

  1. Smaller add-ons. Since they no longer need to ship the APIs themselves add-ons only have to include the unique code that makes them special. That’s something like a 65% file-size saving for the most popular SDK based add-ons, probably more for simpler add-ons.
  2. Add-ons will stay compatible with Firefox for longer. We can evolve the modules in Firefox that add-ons use so that most of the time when changes happen to Firefox the modules seamlessly shift to keep working. There are still some cases where that might be impossible (when a core feature is dropped from Firefox for example) but hopefully those should be rare.

To take advantage of these benefits add-ons have to be repacked with a recent version of the SDK. We’re working on a plan to do that automatically for existing add-ons where possible but developers who want to get the benefits right now can just repack their add-ons themselves using SDK 1.14 and using cfx xpi --strip-sdk, or using the next release of the SDK, 1.15 which will do that by default.

May 15, 2013 10:31 PM

Margaret Leibovic (margaret)

New Coding Stewards Mailing List

As part of the Grow Mozilla effort, the coding stewards have been working to grow our coding community. Our main focus has been to increase the number of contributions to the core mozilla-central codebase, which includes making it easier for newcomers to get started, as well as keeping existing contributors engaged. Recent work has been focused on things like mentored bugs and recognizing contributors.

We hold open weekly meetings every Wednesday to discuss progress on these goals, but we want to give more developers the opportunity to get involved with this effort, so we created the coding-stewards@mozilla.org mailing list for broader discussion. If you’re interested in growing our coding community, but you might not be interested in another weekly meeting, please join the list!

May 15, 2013 06:18 PM

May 14, 2013

Tim Taubert (ttaubert)

Scotland.JS 2013 in Edinburgh

I got to spend Wednesday through Friday in Edinburgh last week to attend Scotland.JS. Edinburgh is a lovely city and I will definitely return to get to know it better. It has great people, beers, food and even a castle - what could one want more?

I arrived on Wednesday, just in time for the TechMeetup. It took place in the Appleton Tower and had about 70-80 attendees with quite diverse technical backgrounds. I had a very interesting talk to a freelance patent attorney and also met a couple of people I knew from recent conferences and meetups in Berlin. After we were out of free pizza and beers we met most of the Scotland.JS attendees in a pub right next to the Appleton Tower. Cue more beers and tech conversations.

Thursday was the first day of Scotland.JS that started with a great keynote from Jan Lehnardt. The most memorable talk of the day was about Functional Reactive Programming using Bacon.js by Philip Roberts. There of course were a lot more great talks but the possibilities of creating and combining event streams kept my head spinning the most. At the end of the day we again went to a neat pub near the venue and I headed to bed rather early, still a little tired from travel and the day before.

Friday started with a short 20-minute walk through Edinburgh’s old town that I needed to pass on my way to the venue. My favorite talk of the second and last day was held by Dominic Tarr that tought us about the internals of leveldb. I also really enjoyed Mark Boas talking about making audio a first-class citizen of the web - a very entertaining and impressive mix of technologies. A GitHub drinkup in the venue’s own pub was a great way to end the conference - lots of good conversations with Scotland.JS attendees and speakers as well as people from the local tech community and visitors of the Scottish Ruby Conf.

I flew home the very next day and already miss Edinburgh with its neat little alleys and pubs everywhere. The JavaScript community has once again shown its value as a very interesting mix of nice welcoming people coming from so many different technical backgrounds. Like after most conferences I feel a little exhausted and my head is full of ideas for future or current (side-)projects. I will try my best to return next year, it has been a lot of fun!

May 14, 2013 06:00 AM

May 08, 2013

Margaret Leibovic (margaret)

Dominant Favicon Color, Revisited on Android

Almost two years, I experimented with using the dominant color of a favicon to give a small icon a colorful background. And over the past week, I wrote some patches to incorporate this design into Firefox for Android!

image

The simple algorithm I wrote long ago was done in JS with canvas, but to use this in our native Android UI, it’s simplest to just do it in Java. Luckily, we already had a dominant color utility method in the tree, but creating a background and border with different saturation levels was trickier than I thought it would be. To solve this problem, I gave the ImageView a background drawable with a solid white interior and a gray border, then applied a transparent version of the dominant color as a color filter. This worked pretty well once I figured out which PorterDuff mode to use, but it made me appreciate the simplicity of CSS.

When testing with various icons, I found that our dominant color algorithm could use some improvement. Our Java algorithm is different than the one I experimented with in JS mainly because it uses the HSV color model as opposed to RGB. Instead of counting every distinct color, we split the range of hues into different bins and find the bin that holds the most colors. For the winning bin, we compute the average H, S, and V values within that bin, and return that as the dominant color. To make sure we only return colorful colors, we ignore transparent pixels, as well as pixels that are close enough to black or white. This simple algorithm may not be perfect, but it works pretty well for us, especially for small favicons.

As an interesting bit of history, after I blogged about my dominant color experiment, Mardak wrote an add-on that used an improved version of my JS snippet. At the time, Wes saw this add-on and incorporated the code into XUL Fennec to create icons for homescreen shortcuts. During the Fennec Native rewrite, Wes reimplemented this feature in Java, and that’s the code I found myself in last week. And for those interested in a more robust solution implemented in JS, there’s actually a toolkit service that does this now.

May 08, 2013 06:37 PM