A Pox On Both Your Package Managers!

I love the package managers available for web developers.  It is such a relief from the pain of copying scripts around a few years ago.  But they also have me quite frustrated now that i am trying to publish some packages i've developed for ESHA Research.  Basically, finding available names to publish under now requires a thesaurus or bad conventions like version numbers in names or ".js" suffixes.

Bower and NPM, all of your packages should have forced producers to use namespaces by default from the start.

Shouldn't this have been obvious?  If you want your package manager to succeed, you need a lot of packages.  If you want a lot of packages, you need a lot of available package names.  People want to use names that are easy to remember and to discover.  There are not many of those that pertain to javascript or web development.  And you can only get so creative with synonyms and clever abbreviations before they become hard to remember and discover.  Therefore, no namespace means fewer packages or packages with worse names.

Namespaces not only create more space for names that are simple and preferred. They create space for reputation development.  People learn to recognize quality sources.  It's good for creators to have their own brand highlighted.  It's good for users to have confidence in the creators.

There are workarounds.  You can prefix all of your package names.  But this doesn't have quite the same effect.  Calling your package "company-package" is a functional solution, but it comes with a price.  It lacks the aesthetic appeal of "package".  Users will gravitate toward the one that appears simpler, cleaner, and lamest of all, quicker to type.  This is not good for the ecosystem.  It can both discourage competitors and promote inferior packages.  Only forcing namespaces upon everyone (like GitHub does) can alleviate such counter-productive biases.

And when they finally come around and add a namespace field to package.json or bower.json, it will have the same problem as the workarounds if they do not force it to be present or automatically fill it in, either from GitHub (home to vast numbers of package sources) or by "doubling" (e.g. package/package).  In fact, the latter is probably the best option.  It makes backward compatibility easy and reverses the advantage early name claimers had by dating them.

However it is done, this change must come.  I've been working in open ecosystems long enough to have seen mistakes like this and the inevitable correction before (Maven).  I just hope it comes sooner than later.  And in the meantime, as i push more packages from our company out there i will probably have to suffer with an "esha-" prefix for the rest of them.  I've already run through a fair bit of the thesaurus for a few of them, with little luck in finding acceptable names.

Remember, kids: NO REPOSITORY IS COMPLETE WITHOUT PROPER NAMESPACING!

Update: Isaac Shlueter was kind enough to point me to a discussion of NPM's global namespacing on their mailing list.  It seems to me now that i have been mistaken to see NPM as a general javascript package repository.  It works as a general package manager, especially with the ability to install straight from GitHub, but as a repository they value discoverability and clarity over simplicity and competition. This limits their usefulness in client development but has, they feel, enhanced it for NodeJS users.  I still suspect this will some day change due to the friction of their intentional scarcity, but we'll see.

Trigger 1.0.0 - Application Events Go Native

Ok, i've written about the trigger library before, but a lot has changed since then. jQuery has gone modular and IE8 has drifted out of my development priorities making native wheels a reality.  In response, trigger.js has changed as well.   It's reached version 1.0.0 and staked out a place on GitHub, NPM and Bower.  The official documentation can now be enjoyed here.

Here's an overview of what's changed is the last year:

• What's New:
• Native support - You don't need jQuery, but if you do use jQuery everything still just works.
• Better sequence control - e.stopSequence(), e.resumeSequence() and e.isSequenceStopped()
• Async sequences - Use e.stopSequence(promise) and when the promise resolves, the sequence resumes.
• Event categories - If event type is a verb, constants (formerly "data") are the object, and tags are the adjectives, event.category is the subject that was missing from the grammar.
• Triggers besides click/enter - You can now declare application events to be triggered by any native (or custom) event via calls like trigger.add('dblclick');
• Special event extensions - You can set up special handlers for particular event types.
• Grunt build
• QUnit test suite

• What's Different:
• tags use # instead of : - the colon was stolen and the tags got a more standard syntax
• event.data is now event.constants - better describes the hard-coded nature of the property
• trigger="foo" is now click="foo" - more triggers required a more flexible attribute declaration

• What's Gone:
• The $.fn.trigger wrapper. With jQuery optional and declarative triggering being the recommended pattern, the byte tax for the manual shortcut was deemed not-worth-it. It's a tiny little jquery.trigger.js extension now.
• IE 6,7,8 support. These don't support custom application events, so you'll need both jQuery and the tiny trigger.old.js extension to make things work in older IE versions.  Oh, and IE 6 and 7 require a JSON polyfill as well.
• e.preventDefault() to stop event sequences. The overlap of meaning was confusing, caused problems in IE9, and couldn't support promises. Use e.stopSequence()!
• jQuery event data == e.data.  e.data became e.constants and application event listeners no longer receive data as additional parameters. Again, the overlap of meaning was a problem and the switch to native events offered no way to pass listeners extra parameters anyway. Support for this is under consideration for jquery.trigger.js, but it would mean triggered event sequences could not be heard outside jQuery's event system.

A Better Storage API

Modern browsers have these very handy shortcuts to persistence called localStorage and sessionStorage.  While they're not recommended for everything due to some feature shortage, they are extremely useful for a modern webapp developer.  They are far better than cookies.

There's just one problem: the API is still really lame. Yes, it's a big jump up from cookies, but that's not saying much.  It's verbose, only works with strings and is decidedly lacking in features and convenience.  This is exactly the kind of wheel that i cannot resist reinventing.

I actually wrote store.js well over two years ago, in its first incarnation.  It has come a long way since then and has been well used here at ESHA Research.  However, it has never yet seen the "light of day" out there in open source.  Now, you can fork it on GitHub!

https://github.com/nbubna/store

store.js handles JSON transparently, provides a very rich API, and is even extensible (with a few nice extensions and a few crazy ones already available in the repo). It supports both concise and explicit function calls to suit your needs (e.g. store('foo') === store.get('foo')).  It even supports namespacing your data with ease.

Check out the README on the github repo for documentation.  I think you'll like it. :)

Interface vs. Implementation

What is more valuable: interface or implementation?

This is the most fundamental question in software development. Actually, it might be the most important question in most engineering endeavors.

I'm here to tell you that it is interface. Implementation is essential, of course, as nothing works without it. And good, fast, reliable implementations are immensely valuable. But while you can have interface without implementation (at least in software), you cannot have implementation without interface. Further, your project will always be judged first and foremost on its interface. This is as true of a simple command line utility as it is of a cutting edge video game. Even you, no matter how efficient and clever your implementation is, are more likely to judge your own software mostly by how simple/natural/intuitive/beautiful you consider the interface to be.

There are exceptions, yes. Tasks that are especially greedy for cycles or memory, big data, and the like. Preference in such things will tilt heavily toward performance. But not absolutely. Once competitors are all within "acceptable" performance standards--whatever those may be--the judgement refers immediately back to interface, with few exceptions.

And interface is harder. Interface is art. Implementation is science. Implementation starts (and usually ends) with a simple pass/fail test: does it work? If need be, you can move on to improving the robustness or performance of the implementation. But these are still simple tests. Does it break on unexpected input? Does this change make it faster or slower? Use more memory or less? Implementation skill is a straightforward matter of knowledge, education, and test code. There are reliable rules, clear choices. Good tests guarantee good implementations.

Interface is about intuition and aesthetics, balancing multiple dimensions of competing concerns. There are no guarantees and no rules, only probabilities and guidelines. There are no tests that can be written to pass judgement on an interface, only users can do that. Even when you are the only user, confident interface design can be a challenge; you are a moving, changing target. What is simple for you today becomes inflexible tomorrow. What is flexible becomes unmaintainable. What is concise becomes conflicting. What is explicit becomes burdensome. What is convenient becomes constricting. What is clear becomes confusing. Assumptions are dangerous, but overdesign can cost you both now and then.

Uncomfortable interfaces are THE impetus for coders to reinvent the wheel, over and over again. If there is a problem with the implementation or its performance, most will submit a patch for it. "Wait," you say, "they can't always get patches accepted. Might that not motivate them to reinvent?" You forget: the patch submission process (or lack thereof) is a feature of any project's interface. Interface is not just the API, it's the entire surface area of the project: name, website, license, documentation, developers, active community, and API. In every case, you'll find an interface is ultimately the problem that created the competitor, though implementation problems do, of course, contribute.

So, if interface is both more important and harder, the moral of the story is...

Prioritize it!

Make it a point to learn the strengths and weaknesses of different strategies for naming and structuring all aspects of a project. Never let implementation dictate, block, overwhelm, or otherwise hinder the interface. Expect problems to arise, both initially and as a result of your solutions to those problems. Never forget KISS, YAGNI, and the like. Never treat such acronymisms (ha!) as anything other than helpful guidelines. Don't write any tests (implementation!) until you have put in some time planning your interface or your interface will be better for testing than actual use. Strive to make your interfaces self-explanatory, intuitive, convenient, simple, flexible, robust, well-documented, consistent, extensible, logical, and familiar. Realize that many of those goals often become opposites when mired in the particulars of the real world. Balance is not always possible. Sacrifices must sometimes be made.

Good luck.

Browser (In)Security

...is a myth for client-side code. I'm not just talking about the fact that all your source code is right there for anyone to see (uglified or not). Your active memory is all available to every javascript debugger on the planet. Your every command to or from the server can be intercepted and/or spoofed. You can't trust anything on the client.

Yet i still see people talk about client security as important. Sanitizing inputs, not executing scripts added via innerHTML, blocking cross origin resource loading, hiding variables in closures.  None of these things actually stop a malicious user (or innocent user with a compromised system) from doing anything and everything they want in the browser.  Seriously, anything they want.

Granted, every one of these things make it harder, and making life harder for the bad guys is a good thing.  I'm not opposed to any of these measures. But while useful and good, none of these browser "security" measures actually make your client-side app safe. The only thing you can actually secure is the server.

Never rely upon your client code to validate, sanitize, prevent or secure anything of importance.  There is no security in the browser. Once it's off your server, it is out of your hands. End of story. Anyone suggesting otherwise is likewise not to be trusted. :)

Dependency

The Java world has Maven and Ivy.  Tools meant to untangle dependency webs, to get you what you need, when you need it and put it where you want it.  I've seen a few attempts over the years to adapt Maven to handle javascript dependencies.  They were abandoned unfinished.  I haven't looked much in the last year.

Javascript development is immature.  Like Java 10 years ago, you generally need to manage the resources manually.  It's up to you to juggle version control, version numbers and untangle the web of dependency.  Once you have your libraries in their proper versions and understand what order you need them in, there are tools to help you execute what you decide/know.  RequireJS, yepnode.js, and Grunt interest me the most so far, but they all have different priorities and all get involved late in the dependency management problem. They help you execute deployment and development, but only after you have manually resolved all your dependencies.

Programmatic resolution is hard. To do it well requires organized public repositories, which require standardized naming, versioning, and dependency declaration. Not to mention bandwidth. The person and/or organization that takes it upon themselves to get this rolling in the javascript world has a lot of work cut out for them. But plenty of reward as well.

Ender does programmatic resolution, and it's package.json could probably be made to do more, but it only seems to run just one level deep, when a deep, tangled dependency web is the inevitable reality of any modular application development. It also neglects script loading, test running, support for app code, and stands as a competitor to other frameworks, not a complement.

Dependency management in the javascript world is harder than Java because javascript doesn't require compilation. Compilation funnels all Java devs through a common build step, making a natural insertion point for a dependency management tool. Javascript lacks that chokepoint. Testing, linting, minimizing, concatenating--all of these have some potential but are still inconsistently used or only used in production deployment, not development. And both deployment and development themselves often are embedded within build/deploy processes for whatever other platforms are being used to build the server side of the web app. Any javascript build process ultimately needs to work as a subprocess of those builds.

Javascript/HTML5 as a platform is still wild, untamed by the build tools and processes that smooth out other application development environments.  What i would really love to have in taming it is a dependency management tool that works with these features:

• Repository support - public would be great, easy tools or conventions for private ones are acceptable. CDNs only get us part-way there; wheel authors need to be able to post libraries.
• Standards for declaring dependency info - to optimize performance and untangle the web this should support/require:
• library name and version
• CDN URLs for the library
• global variables exposed by the library
• the libraries dependencies
• name
• version (range)
• when needed (execution, document-ready, window-load, optional, etc)
• license/copyright info
• Conditional script loading (for polyfills)
• Easy, declarative switching between development (test and lint, but no minimization or concatenation) and deployment (minimize and maybe concatenate)
• Plugins for easy integrating with other build tools (Ant/Maven/etc)

Support for non-browser javascript and things like passing modules into each other (to avoid global exposure) are bonuses.  I don't care about them currently, but other people will.

A large portion of the code seems to be out there.  Grunt build process, Ender dependency resolution, yepnope script loading, and you're already well on your way. If you build it, they will come. Well, i certainly will! I'd built it myself, but there are only so many hours in a day.


#javascript

Readable Code

Code needs to be readable.  Readable code is debuggable, re-inventable and extendable.  But not everyone seems to know what makes code readable.

Occasionally, programmers will use comments to explain their code.  Comments certainly can help make a section of code understandable, assuming the comments are intelligible and up to date with the code.  But they in no way make code itself readable.

    Comments should only ever be used to say things that the code itself cannot be made to say.

In other words, you can use comments to explain why a piece code is there, but not to explain what the code is doing.  Yes, there are legitimate exceptions to that explanation, but again, those all fall under the first statement.

Mindful of the value of self-documenting code, many programmers create code like this:

AddressMapper.prototype.setInitialLocation = function(initialLocation) {
    this.initialLocation = initialLocation;
};
AddressMapper.prototype.setContainerElement = function(containerElement) {
    this.containerElement = containerElement;
};
AddressMapper.prototype.renderMapWithEffect = function(effect) {
    //magic!
};
// and so on...

After all, this is self-documenting, understandable code.  No one can accuse them of being opaque or obscure, right?  Beware the evils of premature optimization, right?  YUI 2 was like this, so it's ok, right? No, it most certainly is not OK.  And YUI 2 made me want to cry.

While long, descriptive names are indeed much better than inscrutable acronyms and abbreviations, they do not always produce code that can be called readable.  The example above, is unreadable because it is repetitive, mind-numbing, finger-cramping blather. When code gets repetitive like this, eyes gloss over and even stupid bugs find it easy to hide. Granted, they are likely to do that no matter what, but code like this does not help. And it gets worse, when you create APIs like this, the app code using it ends up just as obnoxious:

var addressMapper = new AddressMapper();
addressMapper.setInitialLocation('Portland, OR');
addressMapper.setContainerElement($('#portlandMap')[0]);
addressMapper.renderMapWithEffect(AddressMapper.Effect.SLIDE_DOWN);

Verbosity is contagious. It spreads. It infects. It sucks the will to live.

When reinventing a wheel, you are not designing an independent feature.  You are creating an API.  Yes, the wheel must be able to roll and so the raw, unadorned functionality of the code is your priority.  But ugly wheels make for ugly vehicles, and no one wants those.

Readable wheel code must avoid needless repetition and be designed to encourage readable app code.  Now, imagine this fictional plugin as a jQuery plugin:

var map = $.fn.map = function(address, effect) {
    return this.each(function() {
       render(this, address, effectFunctions[effect]);
    });
};
function render(el, address, effectFn) {
    // magic!
};
var effectFunctions = {
    slideDown: function(){...},
    ...
};

Which leads to user code like this:

$('#portlandMap').map('Portland, OR', 'slideDown');

<div id="#portlandMap"></div>

The repetition has dropped to tolerable levels, but it's not exactly a picture of readability.  Let's take it a step further and follow all the Rules for Reinvention:

var map = $.fn.map = function() {
    return this.each(map.render);
};

map.render = function(element) {
    var $el = $(element),
        address = $el.attr('address'),
        effectFn = map.effects[$el.attr('effect')];
    // magic!
;
map.effects = {
    slideDown: function(){...},
    ...
};

$(document).ready(function() {
   $('map').map();
});

<map address="Portland, OR" effect="slideDown"/>

At this point, the plugin code is not only more readable, but the app code has become pretty much ideal--declarative, minimal and extremely readable.  In the end, this should always matter most.  Sometimes, for whatever reason, there will be an irreducible amount of complex, ugly code.  In such cases, you should always strive to absorb that mess into your library and keep your app code clean and neat.

                           A readable API takes priority over readable internal code.

In fact, the lower "level" a library runs at, the less readability matters (and the more test code does!)  Since wheel code is just one small step away from app code, your wheels should still be highly readable wherever possible.