What's wrong with extending the DOM now?

Before you read this post, you should definitely read this post, in which kangax skillfully explains the risks and pitfalls of extending "host" objects, particularly as experienced by the Prototype library. It both proceeds and concludes with strong warnings against the practice, allowing for it only in careful polyfills and specific, "controlled" environments.
It was good advice, but things have changed a great deal in the last three years. The DOM is no longer the dark and dangerous world of cross-browser challenges that it was. It's time to start re-evaluating the risks and benefits:

Past Risks Of Extending The DOM

Lack of specification (prototype exposure not guaranteed)

Guarantees and specifications are important. But the reality is that prototype exposure for the DOM has become remarkably consistent and ubiquitous in modern browsers. As long as you are willing to sacrifice the rapidly fading market share of IE < 9 and the like, there is little to fear here.

Host objects have no rules (unpredictable interactions)

This argument was always weak and gets weaker as browsers (especially IE) rapidly improve, because even when they weren't improved, those creating wrapper libraries also had to watch for this. To say, "extending DOM objects is kind of like walking in a minefield" is to say the same about directly using the DOM, whether or not you extend it. In any case, all but one of the examples are for IE, presumably older versions. The remaining example (overriding target on events) is hardly difficult to avoid. I'm not even sure why you'd want to do that.
The bottom line is that these particular minefields have grown dramatically less dangerous to navigate as vendors have been working to clear them and everyone learns the trouble spots. It never hurts to step lightly when you leave the well-trod paths, but by all means, it's time to relax a little here.

Chance of collisions (hard to scale APIs safely)

IE8 and earlier conflated element properties and attributes, as IE8 disappears, so do the problems this raised. However, the browser is admittedly becoming a richer environment, more features, more libraries, more API surface occupied. But the expansion is not even; some notably different naming patterns have evolved. Specifically, those developing the DOM tend to choose names quite unlike those developing with the DOM.
The querySelectorAll function is probably the perfect example of the difference in naming philosophy. In most JavaScript frameworks, this is wrapped to be something that values conciseness or readability over technical descriptiveness. We would use something like find, a name that i'd wager is extremely unlikely to ever conflict with future DOM specifications. Similar safe names are usually easy to find, and future DOM features are not often hard to avoid. Of course, technical descriptiveness and readable conciseness most certainly do overlap, but when they do, it almost always means that the DOM feature is what you need (or close enough). Just limit yourself to a conditional polyfill in such cases. I won't deny that it can be a bit of a dance, but it is a very easy one to learn.
The other conflict area is with named forms and named fields therein. Named forms are automatically exposed on the document object. This is, in my opinion, still a very good reason to avoid extending the document object itself, at least not with any single-word name, as those are more likely to be form names. Named form fields being made available on the form element itself is a much more difficult challenge. However, this is no more a problem for DOM extenders than it is for wrappers. Try calling hide() on a jQuery-wrapped <form><input name="style"> </form> and you'll see what i mean. No matter what approach you use, you must always take special care and consideration when interacting with a <form> element. Here there be dragons!

Performance overhead (manual extension doesn't scale)

Since we are addressing the risk for modern browsers and kangax's discussion of manual extension presumes the technique to be a workaround for older browsers with poor DOM prototype exposure, it is tempting to dismiss this out of hand. However, manual extension is still a legitimate technique for any library trying to "step lightly" and not employ the blanket coverage provided by prototype extension. The good news here is that when you are stepping lightly, you are extremely unlikely to be extending so much or so often that you need to worry about performance. The modern DOM needs far fewer extensions than it used to, especially if you try to restrain your extensions to powerful utilities instead of clusters of simplistic aliases.

IE DOM is a mess (memory leaks and attr/prop confusion)

The conflating of properties and attributes stopped after IE8. And the famous memory leaks were largely resolved earlier than that. Moving on...

Browser bugs (known flaws in IE8 and older)

Bugs will always be there for those stuck in the past, but in the present, they are closely hounded Wrappers and extenders alike ought still keep a sharp eye out, especially in bleeding-edge features, but the core DOM APIs are now regularly battle-tested by hordes of barbarian code all over the planet.

So What Risk Is Left?

Not too much. I believe the list of risks particular to DOM extenders has been reduced to these very manageable risks:

• New code being opened in old browsers<
• Name conflicts with future/proprietary DOM enhancements<
• Name conflicts with named forms<

Dealing with older browsers in this case means not falling back to increasingly desperate or risky measures but directing them to upgrade instructions or a reduced-functionality version. Well-established techniques abound: conditional comments, Modernizr, even server side response changes are legitimate.
Dealing with future/proprietary enhancements mostly just requires checking before extending the prototypes or the elements themselves. Easy peasy, just remember to do if (!('name' in object)) instead of if (object.name). And you may be surprised by how thoroughly you can avoid trouble when extending objects, if you feel so inclined.
Dealing with named forms means you avoid extending the document object, and when you break that rule, never use any name a form might have (i.e. foo() is dangerous, getFoo() is less so).

Benefits of Extending The DOM

Rich API That's Getting Richer

The modern DOM is a rich API, as capable as its wrappers, if not as concise. And it's getting richer all the time. Those writing wrappers rely on plugin authors to keep up with new features being added to the DOM, or simply leave it to application developers to unwrap the DOM when they need access. Extending the DOM gives you everything the DOM has to offer today, no waiting for new releases or new plugins. Yes, it doesn't protect you from functions that lack cross-browser success yet, but the need for this diminishes every month as more and more users switch to evergreen browsers. Where features are missing, polyfills tend to appear in the wilds of GitHub much sooner than they do in wrappers like jQuery.

Educating Developers

When application developers use jQuery, it is rare that they learn anything but jQuery. They tend to think in jQuery and put all their plugins on the $ whether they need to be or not. They rarely learn to appreciate the wealth of knowledge out there that is not at api.jquery.com. While they're working actively on the DOM, they know very little about it. Libraries that extend the DOM enhance it instead of hiding it away, enhancing developer's familiarity with the DOM. More developers knowing the DOM, means more developers ready and able to fix bugs, innovate new features, and generally push things forward in the browser environment.

Trim The Fat

Wrapper APIs can be fairly small, zepto sneaks under 10Kb when you minify and gzip it. But a part of that 10Kb is devoted to just renaming the DOM interface, rather than adding features. Any way your slice it, wrapping one API with a completely alternate API comes at a cost. That cost is small when the API being wrapped is flawed and inconsistent, but as the flaws and inconsistencies get ironed out, the cost becomes ever less worthwhile. The bandwidth headaches of the burgeoning mobile world only make the extra weight more of an issue.

Conclusions

Best. DOM Ecosystem. Ever.

The "evergreen browser" is ascendant and their market share is wonderfully fractured, with IE 6/7/8 plunging to joyful depths. I can't recall the vendors ever being so motivated to concern themselves with compatibility. And where they do fail in that, those failings are often patchable with polyfills and shims or even some transpiler magic. This is the age of plentiful answers on stackoverflow. This is the age of simple patching and forking on GitHub. Our client side developer tools, editors and command line have never been so capable. Package managers are crowded with assets ready to be installed. It is time to stop being afraid of handling the DOM.

Always Room For Improvement

The DOM is not always pleasant to use. It is not always concise or convenient. There remains a healthy need for libraries to improve upon it. But as IE8 fades into blessed oblivion, it is time to acknowledge that much of what was wrong with extending the DOM in the past has faded also. When you get an urge to smooth those rough spots, add features, and maybe even offer some more concise interface for the DOM, don't be afraid to consider doing it directly, without a wrapper. Because a lot has changed in the last four years, and it only looks to be getting better.

Two Valid, Incompatible Approaches

The prototypes are exposed and you can extend them. This should outperform wrappers and manual extensions, but does raise the risk of conflicts somewhat. Don't do this until you know the DOM (it's present variations and future direction) well enough to choose names that work well to avoid conflicts.
If you are more conflict-shy than performance-focused, you can step lightly and create libraries that only extend when and where they are used and no more, preferable with a small number of additional properties too. Such libraries would not be worth the penalty for DOM-intensive applications but may be able to achieve features prototype-based DOM extenders never could.
I see great value in both techniques, but for different situations. It would probably not even be difficult for some libraries to create similar, parallel versions for the different use cases. Mixing the two in a single implementation does not (for the moment) appear to be worthwhile.

The Deciding Factor

Can you afford to deny (full) support to IE8 users? Windows XP stops getting security updates in a matter of months, and it's approaching demise is helping to purge IE8 from the ecosystem. But that may take a while, and meanwhile, IE8 still carries just under 10% of the market. Whether that number is high enough to avoid DOM extenders is not a question with a single answer. Google has cut off IE8 support already and their next platform won't even support IE9, while some businesses still demand IE6 support for web applications. Do you want to press the web forward or pander to stubborn legacy users? Every choice is a trade off.
For myself, i am looking toward the future. I won't rule out supporting some number of features on older browsers, but i will not use them as a starting point or baseline. It is far more important that my applications work well in the exploding mobile device market than the shrinking IE8 market. Limited resources dictate prioritizing growth over legacy. The extra bytes that come with the one-version-fits-all approach are not appreciated by mobile users, especially international ones. So, those with antiquated browsers will get the browsehappy.com treatment until sufficient resources are available to deliver a functional version specifically for them. I may not always design UX mobile first, but it now seems obvious to approach platform decisions from that perspective.

Time To Go Native With HTML(.js)

In the last few months, i have been drifting away from jQuery, slowly, quietly. Removing dependencies on it in my "wheels", getting my brain down a level and into the native DOM. A few weeks ago, there came a watershed moment; i learned of Voyeur.js and inspiration struck. I realized that jQuery might just be in my way...

jQuery is a wrapper, a completely alternate API to the native DOM. It's a great API, no question, but it has begun to feel a bit unnecessary. The native DOM, after all, is far richer and less troublesome than it ever was back when i first fell in love with jQuery. Voyeur, by contrast, wraps nothing. It simply enhanced the DOM, a little here, a little there. It gave a tasteful sprinkling of ES5 sugar to a C++-style interface that happens to be exposed to a JavaScript environment. Sure, the dot-traversal of nodes by tag names using lazy getters was a delightful hack, but that wasn't what grabbed me. The inspiration was the idea that a few little additions to the DOM might be all it would take to make it feel comfortable to "go native".

But, Voyeur seemed centered on the dot-traversal of the document body. The API it gave offered no way to work on the document head, and the .create and use() features felt a bit unrefined, even awkward. Hardest of all for me to accept was that the code was not ripe for hacking, for extending, for enriching. So, i reinvented it and turned it into a JavaScript library called HTML.

I know what you're thinking. "HTML"? Seriously? The name is taken; it's un-Google-able; it lacks the flavor and pizazz of "Voyeur".  But it has two things i love; two things i cherish:

1. It reads right in my code. The library starts on the root element <html> instead of <body> (like Voyeur). It is the root element. And it is all about HTMLElements. There is no name more readable and self-descriptive that i could give it.
2. It constrains my code. If i named it something fancy like "Pizazz", it could become anything. I don't want that. A functional name centers and focuses development on a single purpose.

But enough about that; you get used to the name quickly. The code and what it enables are what excite me. Check out the demo and the API to get some hints at what you can accomplish. Pay close attention to what the each() function can do. It's the heart of the whole "befriend the DOM" motto for the project, as it makes it oh-so-easy to get/set/call DOM properties and functions on one or more elements. Unless you get crazy with the field aliases, the property syntax is composed of native DOM API, so their documentation is HTML's documentation.  It keeps things lightweight, consistent and future-friendly, as browser makers push new features.

Considering the ease of working with HTML thus far, i think my days of using jQuery are numbered. Right now, HTML works in all browsers IE9 and better.  Even the quickly diminishing group of IE8 users can be supported by conditionally including things like es5-shim/es5-sham and this lovely polyfill. I should, of course, point out that the dot-traversal portion of HTML is not polyfillable at the present, but everything can be made to work.  I think for new developers who are not required to prioritize antiquated versions of IE, including jQuery's large alternative API will very soon no longer be worth the KB.  And i am hoping that this little (2KB) HTML(.js) will help make that transition easy.

Yes, my fellow developers, the native DOM can be your friend!

grunt-init-vanilla

I've been unhappy with the default templates available for grunt-init.  None of them were a good fit for a "vanilla" javascript library like this.  So, of course, it was time to strike out and make a template that did work for me:

http://github.com/nbubna/grunt-init-vanilla

This serves my needs much better and was a good further exercise developing with and on Grunt.  I'm pleased by how much Grunt can do, but creating a template did feel more difficult than i'd expected, even with the available examples.  Much of this comes from a lack of documentation and some surprising results (like the behavior of init.writePackageJSON).

If i had all the time in the world, i'd be tempted to make a grunt-init-init template for creating templates. But that's not gonna happen.  Odds are better that i'll be forking grunt and/or grunt-init and submitting pull requests one of these days.

Just In Case

In my continuing experiments with Grunt, i decided to reinvent a wheel with Grunt from the start, instead of adapting an existing javascript library to a Grunt build.  For no particular reason, i wrote a library to identify and convert the case of strings.  Here's the result:

http://github.com/nbubna/Case

It handles all the usual suspects and is easily extend to handle new ones.  Not a bad day's work.

A Pox On Both Your Package Managers!

I love the package managers available for web developers.  It is such a relief from the pain of copying scripts around a few years ago.  But they also have me quite frustrated now that i am trying to publish some packages i've developed for ESHA Research.  Basically, finding available names to publish under now requires a thesaurus or bad conventions like version numbers in names or ".js" suffixes.

Bower and NPM, all of your packages should have forced producers to use namespaces by default from the start.

Shouldn't this have been obvious?  If you want your package manager to succeed, you need a lot of packages.  If you want a lot of packages, you need a lot of available package names.  People want to use names that are easy to remember and to discover.  There are not many of those that pertain to javascript or web development.  And you can only get so creative with synonyms and clever abbreviations before they become hard to remember and discover.  Therefore, no namespace means fewer packages or packages with worse names.

Namespaces not only create more space for names that are simple and preferred. They create space for reputation development.  People learn to recognize quality sources.  It's good for creators to have their own brand highlighted.  It's good for users to have confidence in the creators.

There are workarounds.  You can prefix all of your package names.  But this doesn't have quite the same effect.  Calling your package "company-package" is a functional solution, but it comes with a price.  It lacks the aesthetic appeal of "package".  Users will gravitate toward the one that appears simpler, cleaner, and lamest of all, quicker to type.  This is not good for the ecosystem.  It can both discourage competitors and promote inferior packages.  Only forcing namespaces upon everyone (like GitHub does) can alleviate such counter-productive biases.

And when they finally come around and add a namespace field to package.json or bower.json, it will have the same problem as the workarounds if they do not force it to be present or automatically fill it in, either from GitHub (home to vast numbers of package sources) or by "doubling" (e.g. package/package).  In fact, the latter is probably the best option.  It makes backward compatibility easy and reverses the advantage early name claimers had by dating them.

However it is done, this change must come.  I've been working in open ecosystems long enough to have seen mistakes like this and the inevitable correction before (Maven).  I just hope it comes sooner than later.  And in the meantime, as i push more packages from our company out there i will probably have to suffer with an "esha-" prefix for the rest of them.  I've already run through a fair bit of the thesaurus for a few of them, with little luck in finding acceptable names.

Remember, kids: NO REPOSITORY IS COMPLETE WITHOUT PROPER NAMESPACING!

Update: Isaac Shlueter was kind enough to point me to a discussion of NPM's global namespacing on their mailing list.  It seems to me now that i have been mistaken to see NPM as a general javascript package repository.  It works as a general package manager, especially with the ability to install straight from GitHub, but as a repository they value discoverability and clarity over simplicity and competition. This limits their usefulness in client development but has, they feel, enhanced it for NodeJS users.  I still suspect this will some day change due to the friction of their intentional scarcity, but we'll see.

Trigger 1.0.0 - Application Events Go Native

Ok, i've written about the trigger library before, but a lot has changed since then. jQuery has gone modular and IE8 has drifted out of my development priorities making native wheels a reality.  In response, trigger.js has changed as well.   It's reached version 1.0.0 and staked out a place on GitHub, NPM and Bower.  The official documentation can now be enjoyed here.

Here's an overview of what's changed is the last year:

• What's New:
• Native support - You don't need jQuery, but if you do use jQuery everything still just works.
• Better sequence control - e.stopSequence(), e.resumeSequence() and e.isSequenceStopped()
• Async sequences - Use e.stopSequence(promise) and when the promise resolves, the sequence resumes.
• Event categories - If event type is a verb, constants (formerly "data") are the object, and tags are the adjectives, event.category is the subject that was missing from the grammar.
• Triggers besides click/enter - You can now declare application events to be triggered by any native (or custom) event via calls like trigger.add('dblclick');
• Special event extensions - You can set up special handlers for particular event types.
• Grunt build
• QUnit test suite

• What's Different:
• tags use # instead of : - the colon was stolen and the tags got a more standard syntax
• event.data is now event.constants - better describes the hard-coded nature of the property
• trigger="foo" is now click="foo" - more triggers required a more flexible attribute declaration

• What's Gone:
• The $.fn.trigger wrapper. With jQuery optional and declarative triggering being the recommended pattern, the byte tax for the manual shortcut was deemed not-worth-it. It's a tiny little jquery.trigger.js extension now.
• IE 6,7,8 support. These don't support custom application events, so you'll need both jQuery and the tiny trigger.old.js extension to make things work in older IE versions.  Oh, and IE 6 and 7 require a JSON polyfill as well.
• e.preventDefault() to stop event sequences. The overlap of meaning was confusing, caused problems in IE9, and couldn't support promises. Use e.stopSequence()!
• jQuery event data == e.data.  e.data became e.constants and application event listeners no longer receive data as additional parameters. Again, the overlap of meaning was a problem and the switch to native events offered no way to pass listeners extra parameters anyway. Support for this is under consideration for jquery.trigger.js, but it would mean triggered event sequences could not be heard outside jQuery's event system.

A Better Storage API

Modern browsers have these very handy shortcuts to persistence called localStorage and sessionStorage.  While they're not recommended for everything due to some feature shortage, they are extremely useful for a modern webapp developer.  They are far better than cookies.

There's just one problem: the API is still really lame. Yes, it's a big jump up from cookies, but that's not saying much.  It's verbose, only works with strings and is decidedly lacking in features and convenience.  This is exactly the kind of wheel that i cannot resist reinventing.

I actually wrote store.js well over two years ago, in its first incarnation.  It has come a long way since then and has been well used here at ESHA Research.  However, it has never yet seen the "light of day" out there in open source.  Now, you can fork it on GitHub!

https://github.com/nbubna/store

store.js handles JSON transparently, provides a very rich API, and is even extensible (with a few nice extensions and a few crazy ones already available in the repo). It supports both concise and explicit function calls to suit your needs (e.g. store('foo') === store.get('foo')).  It even supports namespacing your data with ease.

Check out the README on the github repo for documentation.  I think you'll like it. :)

Interface vs. Implementation

What is more valuable: interface or implementation?

This is the most fundamental question in software development. Actually, it might be the most important question in most engineering endeavors.

I'm here to tell you that it is interface. Implementation is essential, of course, as nothing works without it. And good, fast, reliable implementations are immensely valuable. But while you can have interface without implementation (at least in software), you cannot have implementation without interface. Further, your project will always be judged first and foremost on its interface. This is as true of a simple command line utility as it is of a cutting edge video game. Even you, no matter how efficient and clever your implementation is, are more likely to judge your own software mostly by how simple/natural/intuitive/beautiful you consider the interface to be.

There are exceptions, yes. Tasks that are especially greedy for cycles or memory, big data, and the like. Preference in such things will tilt heavily toward performance. But not absolutely. Once competitors are all within "acceptable" performance standards--whatever those may be--the judgement refers immediately back to interface, with few exceptions.

And interface is harder. Interface is art. Implementation is science. Implementation starts (and usually ends) with a simple pass/fail test: does it work? If need be, you can move on to improving the robustness or performance of the implementation. But these are still simple tests. Does it break on unexpected input? Does this change make it faster or slower? Use more memory or less? Implementation skill is a straightforward matter of knowledge, education, and test code. There are reliable rules, clear choices. Good tests guarantee good implementations.

Interface is about intuition and aesthetics, balancing multiple dimensions of competing concerns. There are no guarantees and no rules, only probabilities and guidelines. There are no tests that can be written to pass judgement on an interface, only users can do that. Even when you are the only user, confident interface design can be a challenge; you are a moving, changing target. What is simple for you today becomes inflexible tomorrow. What is flexible becomes unmaintainable. What is concise becomes conflicting. What is explicit becomes burdensome. What is convenient becomes constricting. What is clear becomes confusing. Assumptions are dangerous, but overdesign can cost you both now and then.

Uncomfortable interfaces are THE impetus for coders to reinvent the wheel, over and over again. If there is a problem with the implementation or its performance, most will submit a patch for it. "Wait," you say, "they can't always get patches accepted. Might that not motivate them to reinvent?" You forget: the patch submission process (or lack thereof) is a feature of any project's interface. Interface is not just the API, it's the entire surface area of the project: name, website, license, documentation, developers, active community, and API. In every case, you'll find an interface is ultimately the problem that created the competitor, though implementation problems do, of course, contribute.

So, if interface is both more important and harder, the moral of the story is...

Prioritize it!

Make it a point to learn the strengths and weaknesses of different strategies for naming and structuring all aspects of a project. Never let implementation dictate, block, overwhelm, or otherwise hinder the interface. Expect problems to arise, both initially and as a result of your solutions to those problems. Never forget KISS, YAGNI, and the like. Never treat such acronymisms (ha!) as anything other than helpful guidelines. Don't write any tests (implementation!) until you have put in some time planning your interface or your interface will be better for testing than actual use. Strive to make your interfaces self-explanatory, intuitive, convenient, simple, flexible, robust, well-documented, consistent, extensible, logical, and familiar. Realize that many of those goals often become opposites when mired in the particulars of the real world. Balance is not always possible. Sacrifices must sometimes be made.

Good luck.

Browser (In)Security

...is a myth for client-side code. I'm not just talking about the fact that all your source code is right there for anyone to see (uglified or not). Your active memory is all available to every javascript debugger on the planet. Your every command to or from the server can be intercepted and/or spoofed. You can't trust anything on the client.

Yet i still see people talk about client security as important. Sanitizing inputs, not executing scripts added via innerHTML, blocking cross origin resource loading, hiding variables in closures.  None of these things actually stop a malicious user (or innocent user with a compromised system) from doing anything and everything they want in the browser.  Seriously, anything they want.

Granted, every one of these things make it harder, and making life harder for the bad guys is a good thing.  I'm not opposed to any of these measures. But while useful and good, none of these browser "security" measures actually make your client-side app safe. The only thing you can actually secure is the server.

Never rely upon your client code to validate, sanitize, prevent or secure anything of importance.  There is no security in the browser. Once it's off your server, it is out of your hands. End of story. Anyone suggesting otherwise is likewise not to be trusted. :)

Dependency

The Java world has Maven and Ivy.  Tools meant to untangle dependency webs, to get you what you need, when you need it and put it where you want it.  I've seen a few attempts over the years to adapt Maven to handle javascript dependencies.  They were abandoned unfinished.  I haven't looked much in the last year.

Javascript development is immature.  Like Java 10 years ago, you generally need to manage the resources manually.  It's up to you to juggle version control, version numbers and untangle the web of dependency.  Once you have your libraries in their proper versions and understand what order you need them in, there are tools to help you execute what you decide/know.  RequireJS, yepnode.js, and Grunt interest me the most so far, but they all have different priorities and all get involved late in the dependency management problem. They help you execute deployment and development, but only after you have manually resolved all your dependencies.

Programmatic resolution is hard. To do it well requires organized public repositories, which require standardized naming, versioning, and dependency declaration. Not to mention bandwidth. The person and/or organization that takes it upon themselves to get this rolling in the javascript world has a lot of work cut out for them. But plenty of reward as well.

Ender does programmatic resolution, and it's package.json could probably be made to do more, but it only seems to run just one level deep, when a deep, tangled dependency web is the inevitable reality of any modular application development. It also neglects script loading, test running, support for app code, and stands as a competitor to other frameworks, not a complement.

Dependency management in the javascript world is harder than Java because javascript doesn't require compilation. Compilation funnels all Java devs through a common build step, making a natural insertion point for a dependency management tool. Javascript lacks that chokepoint. Testing, linting, minimizing, concatenating--all of these have some potential but are still inconsistently used or only used in production deployment, not development. And both deployment and development themselves often are embedded within build/deploy processes for whatever other platforms are being used to build the server side of the web app. Any javascript build process ultimately needs to work as a subprocess of those builds.

Javascript/HTML5 as a platform is still wild, untamed by the build tools and processes that smooth out other application development environments.  What i would really love to have in taming it is a dependency management tool that works with these features:

• Repository support - public would be great, easy tools or conventions for private ones are acceptable. CDNs only get us part-way there; wheel authors need to be able to post libraries.
• Standards for declaring dependency info - to optimize performance and untangle the web this should support/require:
• library name and version
• CDN URLs for the library
• global variables exposed by the library
• the libraries dependencies
• name
• version (range)
• when needed (execution, document-ready, window-load, optional, etc)
• license/copyright info
• Conditional script loading (for polyfills)
• Easy, declarative switching between development (test and lint, but no minimization or concatenation) and deployment (minimize and maybe concatenate)
• Plugins for easy integrating with other build tools (Ant/Maven/etc)

Support for non-browser javascript and things like passing modules into each other (to avoid global exposure) are bonuses.  I don't care about them currently, but other people will.

A large portion of the code seems to be out there.  Grunt build process, Ender dependency resolution, yepnope script loading, and you're already well on your way. If you build it, they will come. Well, i certainly will! I'd built it myself, but there are only so many hours in a day.


#javascript

Readable Code

Code needs to be readable.  Readable code is debuggable, re-inventable and extendable.  But not everyone seems to know what makes code readable.

Occasionally, programmers will use comments to explain their code.  Comments certainly can help make a section of code understandable, assuming the comments are intelligible and up to date with the code.  But they in no way make code itself readable.

    Comments should only ever be used to say things that the code itself cannot be made to say.

In other words, you can use comments to explain why a piece code is there, but not to explain what the code is doing.  Yes, there are legitimate exceptions to that explanation, but again, those all fall under the first statement.

Mindful of the value of self-documenting code, many programmers create code like this:

AddressMapper.prototype.setInitialLocation = function(initialLocation) {
    this.initialLocation = initialLocation;
};
AddressMapper.prototype.setContainerElement = function(containerElement) {
    this.containerElement = containerElement;
};
AddressMapper.prototype.renderMapWithEffect = function(effect) {
    //magic!
};
// and so on...

After all, this is self-documenting, understandable code.  No one can accuse them of being opaque or obscure, right?  Beware the evils of premature optimization, right?  YUI 2 was like this, so it's ok, right? No, it most certainly is not OK.  And YUI 2 made me want to cry.

While long, descriptive names are indeed much better than inscrutable acronyms and abbreviations, they do not always produce code that can be called readable.  The example above, is unreadable because it is repetitive, mind-numbing, finger-cramping blather. When code gets repetitive like this, eyes gloss over and even stupid bugs find it easy to hide. Granted, they are likely to do that no matter what, but code like this does not help. And it gets worse, when you create APIs like this, the app code using it ends up just as obnoxious:

var addressMapper = new AddressMapper();
addressMapper.setInitialLocation('Portland, OR');
addressMapper.setContainerElement($('#portlandMap')[0]);
addressMapper.renderMapWithEffect(AddressMapper.Effect.SLIDE_DOWN);

Verbosity is contagious. It spreads. It infects. It sucks the will to live.

When reinventing a wheel, you are not designing an independent feature.  You are creating an API.  Yes, the wheel must be able to roll and so the raw, unadorned functionality of the code is your priority.  But ugly wheels make for ugly vehicles, and no one wants those.

Readable wheel code must avoid needless repetition and be designed to encourage readable app code.  Now, imagine this fictional plugin as a jQuery plugin:

var map = $.fn.map = function(address, effect) {
    return this.each(function() {
       render(this, address, effectFunctions[effect]);
    });
};
function render(el, address, effectFn) {
    // magic!
};
var effectFunctions = {
    slideDown: function(){...},
    ...
};

Which leads to user code like this:

$('#portlandMap').map('Portland, OR', 'slideDown');

<div id="#portlandMap"></div>

The repetition has dropped to tolerable levels, but it's not exactly a picture of readability.  Let's take it a step further and follow all the Rules for Reinvention:

var map = $.fn.map = function() {
    return this.each(map.render);
};

map.render = function(element) {
    var $el = $(element),
        address = $el.attr('address'),
        effectFn = map.effects[$el.attr('effect')];
    // magic!
;
map.effects = {
    slideDown: function(){...},
    ...
};

$(document).ready(function() {
   $('map').map();
});

<map address="Portland, OR" effect="slideDown"/>

At this point, the plugin code is not only more readable, but the app code has become pretty much ideal--declarative, minimal and extremely readable.  In the end, this should always matter most.  Sometimes, for whatever reason, there will be an irreducible amount of complex, ugly code.  In such cases, you should always strive to absorb that mess into your library and keep your app code clean and neat.

                           A readable API takes priority over readable internal code.

In fact, the lower "level" a library runs at, the less readability matters (and the more test code does!)  Since wheel code is just one small step away from app code, your wheels should still be highly readable wherever possible.

More Fun With Custom Events - state.js

My web apps these days are typically a single page, perhaps excepting a login page or some help content.  One page, composed from many files--during development, at least--does most of the work.  This design obviously began with the usual problems of lacking browser history control and bookmark-ability.  Looking at solutions others made, as you might expect, prompted me to reinvent another wheel.

Lo, state.js:

This enables browser history and bookmarks, by shoving simple key/value pairs into either the query string or the hash, as supported.  Keeping everything in the hash for all browsers would have been simpler, but then it would not survive a server-side login redirection process, undermining the ability to effectively bookmark a gated app.

state.js keeps track of the key/value pairs in memory, in order to notify listeners when those values are changed by the browser/user.  You can register a listener directly with state('foo', listenFn) or just watch for 'foo.state' events on the document.  This latter feature is part of the most interesting feature of state.js: it can be used entirely via custom events.  You do not need to ever directly reference the 'state' function in your app code or other js libs.  Granted, if you want soft dependency like this, it is easy to just have your config code wait for document ready and check if 'state' is available, but as state.js is such an event-oriented bit of code to begin with, i found it useful, amusing, and trivial to add an event-based API.

As an example, this:

state('foo', function(val, oldVal) {
    console.log('foo changed from ',oldVal,' to ',val);
});
state('foo', 'bar');// set foo to bar

is equivalent to this:

$(document).on('foo.state', function(val, oldVal) {
    console.log('foo changed from '+oldVal+' to '+val);
})
.trigger('state', ['foo','bar']);// set foo to bar

And here's the other useful features:

var all = state();// get a copy of current state
var foo = state('foo');// get current 'foo' state
$(document).trigger('back');// tell the browser to go back a page
$(document).trigger('forward'); // === history.forward()

As you can probably guess, this event-based API comes in quite handy when using trigger.js.  Now your page's controls can control state values and shift browser history:

<button trigger="nextPg state['pg',2]">Next</button>
<button trigger="back">Back</button>

HTML Validators Considered Harmful

There are these things out there called "HTML validators".  The last time i tried one was about 12 years ago.  There's a reason i haven't gone back.  It's not that i don't believe in standards; i do.  It's not that i'm sloppy or sneer at perfectionism; i don't!  I don't use HTML validation, because it doesn't work, won't work and shouldn't work.

HTML validation doesn't work because the browsers have never even come close to uniform standards coherence.  So the validators are testing markup against pure fiction, an idealistic but flawed dream of W3C hegemony.

HTML validation won't work because both HTML and browser versions are toast.  Over and done.  HTML has officially been declared what it always was: a fuzzy target.  Browsers have begun a version number war.  It is accepted dogma in web app development that you are doing it wrong when you check the user-agent string for any browser or version number.  Feature detection is the new game, and javascript is now used to emit so much markup into the document that HTML validators are a hopeless cause.

HTML validation shouldn't work, because it has always been worse than useless. It is constricting, a bane to innovation.  Why should app developers and wheel reinventors be restricted to a generic set of elements and attributes?  What good does it accomplish to place such a limit?  Like many other languages, HTML/XML is designed to be extended.  That creative ability should not be left only to the W3C and browser vendors.

In practical terms, browsers are spectacular at one thing, above all else: ignoring things.  They ignore extra whitespace, unrecognized elements, custom attributes, and even unclosed tags in many cases.  They do this so well out of necessity. Standard specifications have never been met.  They also need not only a high degree of backward compatibility, but forward compatibility as well.

Browsers are designed for new things.  Be creative!


The only valid kind of HTML validation is syntax validation.  All i ever want to know is if it is parse-able.  Really, that's it!

Another Short Cut - key.js

Keyboard shortcuts are a wheel that just begs for reinvention.  And being the sucker i am, i gave in and wrote me a new wheel just for that.

Like trigger.js, this is about better events and giving page controls control.  Jump below the gist for explanation:


This does two things:

• allows keyboard shortcuts to be assigned in HTML
• triggers namespaced, natural language 'key' events

<a key="alt-r" href="#reply">Reply</a>
<button key="home">Start Over</button>

$('#foo').on('key.ctrl-delete', Foo.deleteAll);

Both behaviors do not by default happen for most "plain" key events (roughly /[a-zA-Z0-9]/) unless you set key.all = true or add a 'key-all' class to your page's root tag (i.e. <html class="key-all">).

For the HTML shortcuts, key.js triggers 'click' events as appropriate. If you need a different event, add a trigger="different" and use trigger.js to translate the click.

key.js also provides a key() function to jQuery to let you trigger key events from javascript, as is quite necessary for testing your key events and shortcuts.

Obviously, if you look at the code, there are a great many keyCode values for which this wheel does not provide an English translation. At this point, 100% completeness seems both difficult (due to OS variation) and largely useless. Almost all "normal" keyboard shortcuts are accounted for, and any developer using this code can easily add more to key.codes. As with most of my wheels, this one is quite adjustable and extensible. Enjoy!

Give Controls Control - trigger.js

When you put a <button> or an <a> or an <input type="image"> into your markup, you intend them to do something, not just sit there and look pretty.  This inevitably means writing code like this:

$('a.blah').on('click', function() {
    console.log('blah');
});

STOP!  You actually meant to use a custom event, right?  Let's try that again:

$('a.blah').on('click', function() {
    $(this).trigger('blah');
});
$(document).on('blah', function() {
    console.log('blah');
});

Ah, much better. Sure, it's longer, but this way you are self-documenting and making your code easier to test, extend and maintain.

But what if it didn't really have to be longer? What if the button itself could be told to trigger the custom event. After all, that ".on('click'...)" business is just boilerplate and not particularly friendly to the world of keyboard accessibility. Well, this is what trigger.js does for you. Instead of setting up the custom event with some boilerplate javascript, we just extend HTML with a 'trigger' attribute:

<a trigger="blah">Blather</a>

$(document).on('blah', function(e) {
    console.log('blah');
});

Now there's no "blah blah blah" boilerplate; i get to focus on what i'm doing more than how i'm doing it.  It gives my controls control, rather than shoving their functions off into a separate file.  And this can be extended to accomplish even more.

But before we talk about the "more", here is trigger.js:


Wait, you say! Why isn't it just like this:
It could be that simple, true. If that's all you need, use that. But it's hard to resist adding a little flair when you are inventing the wheel. The first thing i wanted was sequenced events.  Why trigger just one when you can trigger more? After all, i like to "use custom events like crazy", and many "actions" in applications are actually combinations of sequential, dependent, but separate actions. Consider:

<button trigger="validate submit">Submit</button>

This way, my "validate" code doesn't have to be tied to "submit" code and so on.  And when "validate" fails, it can still call e.preventDefault() to keep "submit" from firing.

Once i had sequenced events in my markup, it was natural and easy to add them to jQuery's trigger function, especially for debugging things in the console.

$('#cancel').trigger('reset back');

That done, i remembered that i have long wanted to have my own custom event properties.  Instead of "moveUp" and "moveDown", i wanted to have "move" events where i could test for "up" or "down" properties, just like i test e.ctrlKey.  So, trigger.js now supports:

<button trigger="move:up next">Up</button>

$('#game').on('move', function(e) {
    var roll = dice.value();
    if (e.up) player.climb(roll);
    if (e.down) player.slide(roll);
    if (player.winner) {
        e.preventDefault();// all done
    }
}).on('next', function() {
    player = player.next();
});

Finally, i occasionally envied the ability to associate data with an event, like you can when calling trigger() on a jQuery collection. So i added that as well:

<button trigger="level[10]">Loud</button>
<button trigger="level[11]">Spinal Tap</button>

$(document).on('level', function(e, level) {
    speakers.goTo(level);
});

And finally, i wanted my apps to be more keyboard accessible. So, trigger.js also listens for 'enter' events on elements that don't naturally convert them to clicks (div, span, a[!href]) and does that for them.

By all means, if these extra features aren't useful to you, just use the simple, four-line version. But if you want to give your document's controls a bit more control, trigger.js is ready to help. :)

Code of Three Kinds

Client-side javascript can be generally be put into three categories:

• Frameworks  ("raw material")
• Plugins          ("wheel")
• App code      ("ford taurus")

The first two are what i often call "libraries".  They are reusable features and functions meant to be shared across projects.  The third is the code that makes your app what it is and is typically useless to others except perhaps as an example or template of how to use the frameworks and plugins.  Even then, it is often so busy and complex that it rarely works as a clear example.

Frameworks

These are the libraries we are required to use (See Use jQuery).  Probably single most distinguishing feature of these libraries is that your webapps can (and should!) pull them off of a CDN.  These have become part of the plumbing of the internet, and the more people who use a CDN for them, the better.  It speeds things up for everyone involved.

Another common reality about frameworks is that they usually offer far more features than any one app will use.  This is to be expected and accepted.  Customizing them to only download what you need defeats the point of common CDNs and all the caching available that way.  Don't worry about the extra code, it's not worth the worry.

Again, DON'T CUSTOMIZE and DON'T REINVENT, at least never a core feature.  These are not wheels, these have become the raw materials of a webapp.  Reinventing or rearranging them one molecule at a time is a waste of time, unless perhaps, you are doing it on behalf of a major tech company that plans to market and fund it all for you.

Plugins

These are what i am calling "wheels".  They are the shims, polyfills, widgets, extensions and other features that are used to build applications.  They are not specific to a particular application but are not likely to be needed or wanted by a majority of applications out there.  These cannot be found on CDNs but are rather all over GitHub and Google Code and hundreds of personal blogs and websites scattered around the internet.  For each polyfill or widget i might want, there are a dozen competing implementations, all of which do either too much or too little or both.  I usually think i could write them better myself and odds are 70/30 that i'm right.  This isn't just arrogance, the reality is simply that they wrote it for a particular app or audience that doesn't exactly line up with me and my app.  Personally, even when i'm wrong about this, i often learn some good things in the process of trying to write my own version or port one over to jQuery.

I will probably freely switch between calling these "plugins", "wheels" and just "js libs" to keep things confusing.

App Code

This is the code that uses the plugins and framework(s) together in a unique way to make your webapp (or just web page) what it is.  Unless your next app is merely a variation on your previous one, then you can expect that none of this will be reused.  And, in general, it is only worth talking about in the context of ways to reduce and simplify it via better plugins and frameworks.  Just as i would never pollute a framework with plugin customizations, i strive to keep my plugins free of app code.

Make sense?

Example Wheel

Here's an example of a typical wheel i write these days:

/**
 * Copyright (c) 2012, Nathan Bubna
 * Dual licensed under the MIT and GPL licenses:
 *   http://www.opensource.org/licenses/mit-license.php
 *   http://www.gnu.org/licenses/gpl.html
 *
 * @version 0.1
 * @requires jQuery 1.7+
 * @optional roll.js
 */
;(function($, window, document) {

    var wheels = window.wheels = {
        init: function() {
            $('vehicle').each(function() {
                var vehicle = $(this);
                vehicle.trigger('addWheels', vehicle.attr('wheels'));
            });
        },
        add: function(e, count) {
            var vehicle = $(e.target).closest('vehicle');
            for (var i=0; i<count; i++) {
                wheels.attachOneTo(vehicle);
            }
            vehicle.trigger('addedWheels', [count]);
            if (!!vehicle.attr('roll')) {
                vehicle.trigger('roll');
            }
        },
        attachOneTo: function(vehicle) {
            // magic happens here
        }
    };

    $(document)
    .on('addWheels', wheels.add)
    .ready(wheels.init);

})(jQuery, window, document);

This is designed to be interacted with via events and DOM configuration.  It uses jQuery to fire and handle events as well as discover <vehicle> elements in the document.  It adds wheels according to the number specified in that element's "wheels" attribute:

<vehicle wheels="3" roll="true">some tricycle markup here</vehicle>

 It uses events to communicate what it is doing for others to listen and attach behaviors.  It even optionally fires a "roll" event once all the wheels are attached that is meant for a possibly non-existant "roll.js" to handle.

If a <vehicle> is added later, the user can give it wheels by simply firing an "addWheels" event on it.

The user of this "wheels.js" code can enjoy its benefits without ever directly accessing the window.wheels object, making his code both simpler and more robust.  But, if he wants to, he can use, override, or wrap any of the functions in wheels, as well as use them directly during a console debugging session.

Rules for Wheel Reinvention

Here's my rules for reinventing "wheels" for client webapp development:

• Never make the user call a 'setup' or 'init' method
• Declare configuration via DOM intuitive, "natural" elements and attributes
• Use jQuery
• Use custom events like crazy
• Expose your functions

Here's the explanations:

Never Make Users Call .init()

...because it's utterly redundant.  This isn't server-side java where you have no idea what namespaces and libraries have been pulled in by your Maven build and everything needs to be tightly managed.  This is the browser, baby!  Client coders frequently bend over backward to avoid including things they aren't using.  If your code finds itself loaded, consider that declaration of intent to be used!  Wait for window load or document ready and go for it!  Don't be shy about it, presence === permission to function!

Declarative, Intuitive DOM Configuration

This is really a logical extension of the first rule.  It does little good to avoid forcing init() calls if your code can't/won't know what to do without specific user instructions.  Remember, we're talking about "wheel" code here, not application code.  These things are meant to be generic and reusable widgets, components, and behaviors.  You shouldn't be writing a new fancy address mapping widget that makes the user call: AddressMapper.mapOnBlur($('input.address')).  Gag me!

Just extend HTML.  Seriously.  Browsers are wildly, amazingly robust markup processors. Running HTML through some fancy spec validation is generally a waste of time.  This is why HTML5 has officially become a moving target and those arcane doctype declarations are going the way of the dinosaur.  Just declare <!doctype html> and stop worrying about having 100% valid HTML.  The platform is developing too rapidly and inconsistently for anyone to be strict about following HTML specifications now.  Don't use invalid syntax, but don't be shy about creating your own elements, attributes and attribute values.

So, instead of exposing AddressMapper.mapOnBlur(), have addressmapper.js go looking for <input type="address" map-on="blur">.  Be creative, literally.  All browsers can handle this.  Both the code for your wheel and your user's app will be cleaner and clearer because of this.  Instead of three files to be coordinated (the wheel, the user's js and the DOM), there are just two.  The brain power and bloat saved here is well worth it.

Use jQuery

MooTools, YUI, Dojo and the like are all fine projects.  But jQuery is both everywhere and much more than good enough.  If you want to port your wheels to a different framework later, fine, but start with jQuery.  If your boss or customer has already committed to YUI, fine, but port to jQuery at your first opportunity.

But beyond all debates about which framework is best, you should not write your own code to do things that these frameworks all do: event binding, DOM traversal, DOM manipulation, etc.  Don't even write your own feature detection.  Use Modernizr.  The real point of this rule is that there are some wheels that you really, really, really should not reinvent.  The window for reinventing jQuery or Modernizr has long closed.  You cannot wisely develop a modern web app without using an established javascript framework. Deal with it.

Use Custom Events

This follows right with the custom attributes and elements business.  If your user's app needs to know when your code has done something, you should trigger a custom event, either on a related DOM element or else on the document.  You should NOT force them to do:

AddressMapper.doneMapping(function(map){ app.canUse(map); }); //BAD!

This violates the "Use jQuery" rule, since this is a textbook perfect case for custom events, a wheel that jQuery already adeptly "invented" for you.  Your AddressMapper should be doing $mapEl.trigger('doneMapping');  Then the user does:

$(document).on('doneMapping', function(e){ app.canUse(e.target); }); //GOOD!

This doesn't save the user keystrokes, but it saves them brain cycles.  They know jQuery's event system and don't have to learn one you reinvented.

On a lesser note, this also makes it easy for other wheels to interact with your wheel without creating a hard dependency.  And extensions like that are where the "like crazy" part comes in.  If it is useful for users or extenders to know when AddressMapper is 'doneMapping', it is probably handy for them to have a 'startMapping' event too.  Then they can configure something like a custom <loading> element to watch for both:

<loading on="startMapping" off="doneMapping>Creating Your Map...</loading>
             <!-- Heh. Now i want to make a loading.js that supports this... -->

The key thing here is this:  Events drive web app behavior.  If you are going to be adding behaviors to a web app, i strongly recommend that you use events to drive them too.

Expose Yourself

Make your functions accessible.  If you are a god among coders and your wheel is thoroughly bug-free, your "internal" functions may still be useful for extenders, coders trying to reinvent your wheel, or maybe even power users who want to add a little spice by wrapping a function.  And if you are human, being able to call internal functions sure helps other people find your bugs.  Heck, it may help *you* debug your code; who knows if anyone else will ever use it. ;)

Write your code to be readable, reusable, and recycle-able; leave the reduction to the minifiers and compressors.

For each wheel you write, you should have a single, descriptively-named object (or function) to which you attach all your functions.  This ideally matches the first portion of the file name.  Do NOT expose all of your functions directly in the global namespace.  Don't share all your variables under your primary object/function either; restrict yourself to the important ones.  You can expose this primary object/function as an attribute of the window, jQuery, or some other variable.  The only goal is to make it accessible to extenders and the console's command line.

Reinventing The Wheel

I used to live in the server-side Java world.  Now i live and breathe client-side javascript.

My primary observation about both so far is that in an environment of fast developing platforms, the frameworks and tools never quite seem to keep up or otherwise meet the needs of even half the developers out there.  When faced with a shortcoming, rather than wait for fixes to be incorporated, there are always plenty of coders who would just rather "do it better."  So they reinvent the wheel, sometimes for better, sometimes not.

I increasingly find myself becoming part of the "problem." I see holes in frameworks and browser APIs and get ambitious ideas about filling them.  Or i go look for other people who filled them already only to find that they don't conform to the ideology about client web apps that has formed in my head over the last five years.

So i'm starting this blog to pontificate about the ideology that has formed thus far, and honestly, to codify it for the benefit of my own brain.  As a verbal processor, the act of setting these ideals outside my brain is of great personal benefit.  These ideals currently take the form of "rules" i intend to promote to the client-side javascript developers of the world.  I will probably also use this to share "wheels" that i have personally bothered to (re)invent for public consumption.  And yeah, there may be the occasional useless rant about something in the client javascript world that currently irritates me.

The rules to follow are like any other code i write: subject to continual revision and occasional reinvention.  Thoughtful feedback is likely to drive that process.  Kneejerk responses and angry ego-bashing are liking to inhibit it.  But you knew that already, right?

Thanks for your patience.  Especially with the "wheel" metaphor, as it is likely to get tired.  Bah-dum- ching!