Microsoft buying Minecraft: a (VR) Home of their own?

It was "officially" announced that Microsoft is buying Mojang, the developer of Minecraft, for $2.5 billion dollars. I haven't seen it documented how much of that is cash, stock, deferred sales/licensing royalties, or what. What I have seen is a lot of people griping on social media about how "aimless" the purchase seems to be for Microsoft. Personally, I think it's a genius strategy, and well worth the amount reported, for three reasons that are too nuanced to convey over Twitter.

First, Microsoft had to do something to slow the PS4's momentum with the holiday coming up. They really don't have a unique way to compete going into the holiday season, since they can't rely on a thundering herd of existing purchasers being influencers in their social networks. Is the alternative to "take content away" from PlayStation via exclusive deals? That hasn't worked out (yet), either. It's been widely reported that many PS4 purchasers are folks coming from the Wii -- which is undeniably positioned as a family-friendly console. Unfortunately, the PS4 doesn't currently have a lot of family-friendly titles at retail -- there's Knack and a couple of (visually underwhelming) Lego games. (Everyone who cares knows there's a TON of family-friendly games available digitally for PS4, but people walking into a Best Buy or Toys R Us will make a decision based on what's on the shelf.)

This holiday will see Little Big Planet 3, SingStar, Just Dance, and Lego Batman 3 available for PS4 (and/or PlayStation TV), which will certainly help drum up interest in parents looking to engage with their children via gaming in a way that tablets and phones simply don't allow. Several parents I know are also getting concerned about their children's eyesight being messed up by having bright iPad screens so close to their face all the time. PlayStation has been marketing Minecraft fairly heavily, paying dearly in shelf space with a retail version for PlayStation 3. For parents souring to mobile for various reasons, but needing a recognizable franchise as a pivot point for themselves (and their kids), Minecraft bundles at retail gives those parents (and kids!) the comfort zone they need to transition to a next-gen gaming console.

The folks at PlayStation are smart, and very likely know that continuing to capture the gamer families transitioning off of Wii and/or iPhone/iPad will be key for expanding the market for PS4 (and Vita). The opportunity is especially massive, given the crater left in the family gaming market as families skip the Wii U in droves. As a result, even though there's been no announcement, I would bet money that there were/are solid plans for Minecraft bundles for PS4 *and* Vita planned for this holiday season. Microsoft's purchase of Mojang potentially handicaps that strategy to a large degree. Sony now has to decide whether they want to follow through build a huge marketing and physical retail campaign around what is now their primary competitor's franchise. This is a genius move on Microsoft, but they are relying on a more Japanese attitude to prevail. If Sony has any vision, they'll stay the course and upsell other local co-op/cross-play PlayStation exclusives in the same bundles: Knack and Resogun on PS4, Guacamelee and LittleBigPlanet on Vita, etc. If Sony doesn't stay the course with their likely plans for Minecraft PS4/Vita bundles, they may miss out on a bunch of consumers flocking away from Nintendo and iOS devices during holiday 2014.

Second, Minecraft gives Microsoft something they desperately need -- the basis (mostly branding-wise) for a persistent, online, social space that will give younger gamers a place to transition to instead of Snapchat, Facebook, and whatever else comes up in the next few years. This is likely to be Facebook's long-view strategy for purchasing Oculus as well -- getting ahead of providing a space for existing youth that will leap-frog Facebook's existing offering. It's no secret (or fairly obvious, at least) that Sony's strategy with their own persistent, online, social space -- PlayStation Home -- moving forward is going to be rooted in coupling it to their Project Morpheus Virtual Reality headset. This makes perfect sense for the psychology of hard-core Home users who have spent thousands of dollars in-game, and will likely plunk down the $400 to retain their content and "level up" in the eyes of their peers on Home. Conservatively, if that strategy pans out and helps give Project Morpheus even a slim majority of its first 25,000 worldwide sales (along with the PS3->PS4 conversion) then the strategy will further secure a foothold for their VR platform. Unfortunately, even early power users of Home (like myself) are unlikely to return -- never mind enticing new users who may continue to view Home as an also-ran despite its re-design for VR. For folks who want to invest in a platform that has a persistent on-line social space they can better relate to (or a perceived lower barrier to entry), Minecraft on a Microsoft platforms may be a better fit for some consumers in contrast with Nintendo's anemic offerings and Sony's probable go-big offering coupled to Project Morpheus in 2015. (This doesn't even account for a centralized, persistent Minecraft social space as an upsell to an Xbox AR/VR helmet. For all it's hardware inferiority, Xbox One could certainly render Minecraft in stereo @ 72hz.) This strategy also jives with Microsoft's new CEO being more about revenue from ongoing service subscriptions.

Third, Microsoft is learning mistakes that Nintendon't. Microsoft is realizing that there is more than one way to win, and repeating their mistake of being late to the post-PC (read: WIntel) party is not going to happen again. Sony has been making mobile companion applications for their games for a few years now (Beyond: Two Souls for co-op play on PS3, Knack on PS4, SingStar 2014 on PS3/PS4), and while I don't know if the availability actually helped upsell more people into the primary platforms in any meaningful quantities, it most certainly didn't hurt. PlayStation Mobile certainly didn't set the market afire with its support of Android phones, but it did give access to minor Sony franchises like Lemmings and Tokyo Jungle to non-PlayStation platforms. As proven by Microsoft's masterful execution of their .NET/C# strategy, they *can be* great at listening to the customers in a given market even better than the competition (Sun/Java, in the case of .NET/C#) and trounce the competition as a result. Minecraft already being incredibly popular on many platforms opens up this avenue for Microsoft -- as a publisher first, and a platform-holder second. If Xbox One continues to languish as consumers continue to purchase the PS4 at incredible rates, then they'll need to start laying the groundwork now for them being a cross-platform (not just PC) publisher. In the same way that Nintendo not publishing even a minor franchise (like Pikmin) on another platform totally cut off a potential avenue for better market awareness and upsell of Wii U, Minecraft continuing to be cross-platform is likely Microsoft's way of not making the same mistake. The fact Microsoft would be doing so with a game-as-a-service that has ever-increasing ongoing revenues as platform penetration goes deep and broad, is really quite smart. In the same way that Nintendo sells a steady number of consoles just by refining Mario and Zelda formulas (or even re-releasing previously released games), Microsoft probably sees Minecraft as their long-tail "Mario" (pun intended) in a world that is potentially going to be post-Halo in the medium-term.

Phew. Those are the three legs of the stool as I see them, and it is quite an impressive and holistic set of possible strategies. In the same way that Sony's hubris early in the PS3 cycle induced near-fatal failure and caused an internal cultural transformation that won them 1 million new PS4 players each month since launch, Microsoft is now having the same revelations. If the Xbox One is reduced to being a Minecraft (+VR) machine, in the same way the Vita is a LittleBigPlanet/Spelunky machine for some, is that really a "loss"? If the steady drip of recurring revenue and long-tail increases in user base make for a good business -- and good will toward the platform? Microsoft appears to be doing all the right things by synthesizing a comprehensive strategy based on avoiding their own failures, but more importantly their competition's successes.

It's worth $2.5 billion, in my opinion.

Pesky Perpetual Problems in Parsing PACER Case Information

In the last year, I worked with a legacy application that automated PACER interactions for users and wrapped it in a nicer user interface. This post is about a few of the the data issues, both in data model and in scraping, that I noted as I worked through dozens of issues -- both major and minor -- with the engineers and contractors I brought on board to help out. The issues mentioned below aren't specific to that code base, so I hope that some general documentation and advice helps people working to truly open up public court data for deep analysis.

 The biggest challenge with PACER is not that each court upgrades their instance of the PACER software distribution separately, but rather that each court can bizarrely customize different aspects of the PACER data itself. This is a bit of a nightmare, because it means that your testing has to have pretty amazing coverage across all the PACER sites and then their cases to try and tease out issues before your data set gets corrupt. All the PACER sites had pretty consistent error messaging on their login screens: wrong password, your bill is overdue, your client code is incorrect, etc. The problem is when an individual case, docket entry, or document has been sealed or otherwise had its access restricted. I saw literally dozens of different error messages, coming at different points in the PACER interaction, as local admins sometimes used the proper configuration features of PACER to do things or just did one-off custom errors that they wrote themselves. The reason this becomes so critical is that the end users of your PACER-wrapping application need to understand the problem is with PACER -- and not your application. After a year of doing test-driven development as we fixed customer- and tester-reported problems with specific PACER cases, we finally reached a point where things were mostly fine. Until another PACER admin puts errors into something other than a <b> or <h4> tag.

The second biggest challenge is when cases change venue. The prefix in the case number might change, but it is still hypothetically the same case and should carry the same accrued metadata along with it. Due to the way ECF data entry works, the link between the cases's original venue and the new one is often obvious, but just as often it is not. The text in the docket entry appears to get hand-edited or the ECF software doesn't close an HTML tag, and sometimes the documents themselves can have their header and footer text modified. The latter issue means that an incredibly basic text recognition and comparison won't work to find "same" documents in other cases. So, if I'm following a case and it moves (I may not know if I'm not involved), I'll now be monitoring an ostensibly dead case. Dealing with this in the data model is challenging, but so are the UI aspects -- how do you reasonably notify a research user following dozens of cases both in-UI and via email alerts?

The third challenge is that the recent upgrades to the PACER system are making more and more use of JavaScript. For companies that are using static page scraping technologies, like Nokogiri or Hpricot, the introduction of PACER pages with even a minor amount dynamic DOM necessitates a dramatic change-up in underlying technologies. Since I worked on web application security products in the early 2000s, I has already participated in the cat and mouse battle between web sites that don't want to be crawled and the folks writing the crawling software. My best advice would be for anyone doing (semi-)automated PACER traversal to start using PhantomJS, perhaps even with Capybara, to automate interactions with PACER. If you're already doing acceptance test-driven development, you're probably already using those technologies for your own test suites and perhaps even know how to keep Capybara/Selenium test suites fast and maintainable in the medium/long-term. For some PACER-related product teams, this eminent problem may be the final nail in their coffin.

With the recent data loss of important public court records on PACER*, keeping an open and secure archive of the data in the existing system is critical -- especially before the "NextGen" upgrade where the individual courts' custom handling of various things will potentially lose information forever.

If enough people like this article, I'll write some more in-depth details on specific courts/cases that were non-trivial to parse and model in the database.

* The Latest Really Big Screw Up With PACER

Browser Wars: August 2014, Console Edition

The web browsers on PlayStation platforms, even the PlayStation 4, are extremely under-performant in most standard benchmarks when compared to pretty much any other device -- even years-older Android phones also made by Sony.

One of the most frustrating things about the PlayStation 3, and now the PSVita and PS4, is the oddly broken web browsers. Seemingly intentionally handicapped as far as HTML5 functionality, they also perform significantly worse than most comparable (and technically inferior) platforms, to the point where many of Sony's own web sites can cause the browsers to be extremely jerky and/or crash. (A good example is just loading up the PlayStation Blog and scrolling down the page -- huge pauses and really jerky even when no games are running.) Indeed, a couple of the people I know who have "jailbroken" their PS3 and Vita cited getting a non-gimped web browser on the devices as a motivator. It's a huge pain to try to look up a walkthrough/trophy guide and need to dig out another device just to do it.

Before getting into it, let me thank my pals Corey and Lloyd for running the tests on their Xbox 360 and Xbox One consoles. Lloyd even made YouTube videos showing Fishbowl's performance at varying levels, with some hilarious textual commentary.

Starting with the PS4 (firmware 1.75) and its 8 cores and as many gigabytes of RAM, there's several easy ways to get it to crash or become so unresponsive you think the system is hung: go to iwaggle3d.net, joystiq.com, or even the PlayStation Blog on some days, and scroll down the page. I even got crashes when trying to watch the new music videos on weirdal.com, never mind that most of the videos wouldn't even play inline or on their natively-hosted web sites (funnyordie.com, nytimes.com, etc).

Then there's the performance of the Fishbowl benchmark, easily the most embarrassing aspect on PlayStation platform browsers. The Vita (firmware 3.18) barely gets 1 fps (you read that correctly), and the PS4 -- with its 8 cores, 8GB of GDDR5 RAM, and plentiful GPU compute cores -- gets as high as 3 fps.  For comparison, the iPhone 3GS (running iOS 5.1) gets ~30fps, iPad 2 (running iOS 7.0.2) gets 60fps, and both the Xbox 360 and Xbox One (both running latest software as of August 2014) get 60fps. If you want to get the frames per second up on PS4 and Vita, un-checking the visual components that use alpha (Back, Shine, Front, etc) can get the Vita up to 3fps and PS4 up to 20fps. It's bizarre that alpha is so incredibly slow, meaning it's not only not GPU accelerated but also severely unoptimized. It's also weird that it's so slow despite the audio not playing: when testing the Fishbowl benchmark on the Sony Xperia phone running Android 2.3, the audio plays just fine. (Looking at html5test.com, PlayStation browsers have support for HTML5 audio but support no codecs -- not even license-free ones like PCM or Ogg, nor licensed codecs already included on every shipping PS4 and Vita such as AAC.)

Fishbowl running on PS Vita, getting 1 frame per second (no audio or background video):

Fishbowl running on Xbox 360, getting 60 frames per second (with audio and correctly looping background video):

Fishbowl running on iPad 2, getting nearly 60 frames per second (with audio, but no background video):

  Fishbowl running on PlayStation 4, getting 2 frames per second (with background video, but no audio):

Fishbowl running on Xbox One, getting 60 frames per second (with audio and correctly looping background video):

One might say that HTML5 Fishbowl benchmark is "too new", despite it being about 5 years old performing pretty well on 5 year old devices and the now-defunct Internet Explorer 9, Firefox 17, and Chrome 20 desktop browsers. (That being said, the Android 2.3 web browser on the Xperia Play didn't play anything -- it just played the audio.) There's also the FishIE Tank benchmark, which just demonstrates CSS animations and transitions without using any HTML5 features. In that benchmark, nearly every browser and device gets 60fps -- except PlayStation platforms, where Vita gets 7-13fps and the PS4 gets ~20fps with the default of 20 fish being animated at once. Set the FishIE Tank benchmark to 50 fish, and both PlayStation platforms' browsers nosedive in performance. The iPad 2 and Xbox 360 get 60fps without even breaking a sweat.

 FishIE Tank benchmark on PS Vita, getting 6 frames per second:

FishIE Tank benchmark on Xperia Play running Android 2.3, getting 13 frames per second:

FishIE Tank benchmark on iPad 2, getting 57 frames per second:

FishIE Tank benchmark running on PlayStation 4, getting 22 frames per second:

Even non-graphical, JavaScript benchmarks continue this trend on the Vita. Looking at Octane v1 benchmark from Google, the Vita gets the lowest score of 15.4 on the Richards test. For comparison, I found the lowest-powered device with the oldest browser I could -- a Kindle 2. The Kindle 2 running the 3.3.x software, gets a score of 51.0 in the Richards part of the Octane v1 benchmark. The abhorrent performance of the Vita browser continues throughout the Octance v1 benchmark (and even into Mozilla's Kraken 1.1 benchmark).

I mention the first test from each benchmark here, but the results are quite consistent across all the tests -- the ones that don't crash the browser, anyway. The Splay test in Octane v1 causes both the PS4 and Vita browsers to run out of memory and make the OS a bit unresponsive, and the Kindle 2 browser also has trouble with that particular test. The iPhone 3GS and iPad 2 also have a bit of trouble, but all those browsers still outperform the Vita in the benchmarks that do complete. The Xbox 360 browser crashes during the Richards test, while the PS3 has decent performance on Richards (2x the Kindle 2) but crashes during the Crypto test. The Android 2.3 browser on the Xperia Play doesn't grok some basic aspect of Octane v1 and redirects to a bogus URL.


Octane v1 benchmark running on Kindle 2, getting 49.8 in the Richards test:

Octane v1 benchmark running on PS Vita, getting 15.4 in the Richards test:

Octane v1 benchmark running on iPad 2, getting 1,742 (yes, 2 orders of magnitude higher than PS Vita) in the Richards test:

Octane v1 benchmark running on Xbox One, getting 150 (10x slower than PS4 and iPad 2) in the Richards test:

Octane v1 benchmark running on PlayStation 4, getting 1,737 (pretty good!) in the Richards test:

The Kraken 1.1 benchmark tests from Mozilla run so slow on the Vita, I thought the device was hung. The Vita browser finishes the ai-astar test in the Kraken benchmark in 167,658ms -- over an order of magnitude slower than the next slowest browser. The Xbox 360 was also incredibly slow, but did complete. The PS3 browser simply crashes with a JavaScript error when trying to run Kraken benchmark.

Kraken 1.1 benchmark running on Xperia Play with Android 2.3, getting 12,195ms in the ai-astar test:

 Kraken 1.1 benchmark running on PS Vita, getting 167,658ms (WTF!?) in the ai-astar test:

 Kraken 1.1 benchmark running on Xbox 360, getting 25,223ms in the ai-astar test:

 Kraken 1.1 benchmark running on iPad 2, getting 3,805ms in the ai-astar test:

 Kraken 1.1 benchmark running on Xbox One, getting 7,902ms in the ai-astar test:

 Kraken 1.1 benchmark running on PlayStation 4, getting 10,010ms in the ai-astar test:

SunSpider 1.0.2 benchmark is the same -- the Vita is ridiculously slow but the PS4 performs comparably with the iPad 2. The Xbox 360 fails to run the 3d-cube test properly, resulting in an artificially low score.


SunSpider 1.0.2 benchmark running on Kindle 2, getting 6,023ms total in the 3D tests:

SunSpider 1.0.2 benchmark running on Xperia Play with Android 2.3, getting 462ms in the 3D test:

SunSpider 1.0.2 benchmark running on PS Vita, getting 4,513ms (order of magnitude slower than Xperia Play) in the 3D test:

SunSpider 1.0.2 benchmark running on Xbox One, getting 194ms in the 3D test:

SunSpider 1.0.2 benchmark running on iPad 2, getting 195ms in the 3D test:

SunSpider 1.0.2 benchmark running on PlayStation 4, getting 221ms (pretty good!) in the 3D test:

The good news? The PS4 appears to have a JavaScript JIT in action, but appears to totally lack CPU optimization or GPU acceleration of any kind. The PS3's browser got a major bump to a new version of Netfront (not NX) a few years ago, and many of the media applications on PS3 and PS4 (Netflix, etc) are really HTML5 applications that are running a better-optimized version of WebKit than the user-accessible browser application. Reasonably, the user-accessible browser application on PS3 should eventually get the same webkit/NetFront NX backend as existing applications.

The bad news? The Vita appears to have a severely unoptimized JavaScript *interpreter*, like khtml (the basis for webkit) used to have 5+ years ago. Technically, the Kindle 2's browser also lacks a JavaScript JIT but it does appear to be optimized. My only guess is that the PS Vita web browser is literally an un-optimized debug build, which is at best an embarrassing oversight on the part of Sony's release engineering. (Perhaps some of the Vita homebrew/reversing community can take a look and verify this.)

The ugly news? html5test.com, like acid3.acidtests.org, only tells part of the story: 1) companies that sell browser middleware (Netfront) puts crazy hacks in to make the basic tests pass to improve the overall score, but applications that try to actually use the features tested for will fail spectacularly; and 2) a feature being present but being extremely under-performant in industry-accepted benchmarks or in pages on the popular web sites is not very useful. (A good example of this is the current PS3 browser appearing to pass ACID2 and ACID3 tests, but mis-rendering the BetaFishIE CSS3 animation demo.)

In the meantime, we can hope that Sony will give the dedicated PS3 browser the same webkit backend as used by the new PlayStation Store application, and that the PS4 and Vita will both get reasonably optimized packages of a refreshed Netfront NX browser soon. Since the toolchains for all the platforms support Link Time Optimization and Profile-Guided Optimization, just recompiling the existing code with a runtime profile of these benchmarks (like Firefox, Chrome, etc do) would likely make an enormous difference in performance while also bringing the binary (and therefore resident memory) size down. Let's hope they do it quickly so that people have one less reason to jailbreak the devices and further open the platforms to increased piracy. It's all about providing value.

Below is a table with some of the relevant benchmark values on different platforms. The Vita performed the same regardless of being on AC or battery power, with and without a fresh boot. The PS4 performed the same whether a game was "suspended" in the background, with and without a fresh boot.

Platform	Fishbowl	FishIE Tank	Octane v1: Richards	Kraken 1.1: ai	Sunspider 1.0.2: 3d
PlayStation 4 (1.75)	2fps	22fps	1,737	10,0010ms	221ms
Xbox 360 (Jun 2014)	60fps	60fps	BROKEN	25,223ms	BROKEN
PlayStation 3 (4.60)	BROKEN	1fps	95	CRASH	334ms
iPad 2 (7.0.2)	57fps	60fps	1,742	3,805ms	195ms
Xperia Play (2.3.4)	N/A	13fps	N/A	12,195ms	462ms
Kindle 2 (3.2.1)	N/A	BROKEN	49.8	N/A	6,023ms
PS Vita (3.18)	1fps	6fps	15.4	167,658ms	4,515ms
Xbox One (Aug 2014)	60fps	60fps	150	7,902ms	194ms
	higher better	higher better	higher better	lower better	lower better

spies versus mocks, 2014 edition

Johnny says to learn to use your debugger, kids. Also, chew your gum.

After 6 years of mostly doing low-level C and C++ agile engineering (along with some personal C# projects), I came back into the Ruby and JavaScript fold last year when I accepted a CTO position. I hadn't paid incredibly close attention to the Rails and front-end JavaScript worlds from most perspectives, but was very much looking forward to how things had improved from a developer practices and tooling perspective. 

One interesting thing is that edit/test cycles are so fast at this point with interpreted (read: non-compiled) languages, that many developers don't really use the debugger. To them, it's easier to iterate 2 or 3 times by adding console.log() or Rails.logger.* calls, re-running the test (sometimes manually), and checking the output. An older engineer I worked with over a decade ago called this practice of incrementally adding temporary log messages as "tombstoning": Tombstones are what you trip over when running through a graveyard. This practice isn't really practical in large C/C++ codebases, where even the fastest of build+test cycles is still in the high double-digit seconds.

Despite the process being fast enough to iterate this way thanks to guard-livereload (and other great tools), I still see the waste of context-switching in this practice -- especially when the cycle involves going from the code editor to a web browser, sometimes having to refresh manually, then manually click through a use case. With the several people I paired with on these Rails, Node.js, and ReactJS+Backbone components over the last year, I had to re-introduce and gently push the practice of using the debugger to set a breakpoint to cut the number of times we had to re-run the test (and context switch between several other developer tools). Chrome, Firefox, and Internet Explorer 11 all have fantastic debuggers (each with their own strengths and weaknesses), and RubyMine's Ruby/Rails debugger integration works *extremely* well in version 6.3.x on MacOS Mavericks. What I often heard is that both JavaScript and Ruby debuggers had *previously* been flaky and/or hard to use, and so people fall back to 1960s style debugging practices.

Wait, isn't this post supposed to be about Strict Mocks versus Spies? Yes, it was -- and it is. The perception of maturity and reliability of debuggers, and tooling in general, plays into a root cause of why many developers I have been working with favor spies over strict mocks. Also, there is seemingly still a general affinity in the Ruby community toward optimizing code for readability, often by leveraging deep Ruby features and optional syntax to create DSLs that read from left-to-right, at the expense or afterthought of other medium-term aspects of developer productivity*. Namely, how efficient is it to figure out what's going on when a test FAILS at runtime.

We have all heard the adage from Bob Martin and Kent Beck that code will be read many more times than it is written, and that is the economy to optimize for. I totally agree, but like many XP practices the principle behind this is a reaction against an anti-pattern of "macho" coding where people are proud of their 280-character one-line of code with no spaces whatsoever. The principle of being mindful of your current and future peers -- and your future self -- with regard to making code, especially tests, approachable and "as simple as possible, no simpler", is an important one. Unfortunately, between the DSL-mania at the end of the last decade and the perception of poor tooling, people focused purely on the static approachability of code and tests and not the runtime aspects.

The nice thing about Spies is that they blend well with existing assertions, after the arranging and invokation of the module/class/function being tested. Consistency is good. The downside is when a Spy assertion like this fails:

--

let (:invitation) { invitation = double('invitation').as_null_object }

before do

  InvistationSender.send(invitation)

end

it "delivers invitations to both recipients" do

  expect(invitation) .to have_received(:deliver).twice

end

--

Let's say the test fails because the invitation.deliver method is called three times instead of the specified two times. Where did the third call come from? At this point, the step might be to to a grep/find in the sources looking for references to the deliver method and then eyeball the code, playing computer in our heads to see where the logic fell down. Now, if we are keeping our changes small and our intervals between test feedback short, then both of those practices woudl reduces the scope we have to manually review. Still, let's say that we just upgraded a Ruby gem, Node.js package, or what have you -- there's not any easy way to slice a common development task like that to be thinner.

Manually reviewing in an instance like this often means trying to follow the call stack down in the IDE and ending up in your third-party gems/modules. This huge context switch is a major productivity killer, and can be even more frustrating if you and your pair were previously cruising along. What we really want to know is, "where did the unexpected call come from?" -- more specifically, "what is the call stack (and conditional branches) that led to the unexpected call?".

This is where strict mocks come in handy: in this instance, when the expected number of calls is exceeded, or a parameter constraint does not match, an exception will be thrown/raised, and we'll get a call stack in our console output. If we need more context, we can just re-run the test again but under the debugger this time. Under the debugger, when the invariant violation for the mock is thrown, we'll be able to switch across the frames of the call stack and inspect local variables in right then. It's possible we'll need to do one more cycle where we set specific breakpoints, but often the cycle ends there -- inspecting the variables in the stack frames and in the global context give us enough information. Also, we reduce the context switches between different tools that have differing visual layouts, key bindings, etc.

--

let (:invitation) { invitation = double('invitation').as_null_object }

it "delivers invitations to both recipients" do

  expect(invitation) .to_receive(:deliver).twice

  InvistationSender.send(invitation)

end

--

So, there's a pretty good case for using strict mocks instead of spies when our test is pinning down the *most* number of calls expected for a mocked method/function or constraining the parameters passed to the mock. What about when we're doing different kinds of constraints, like saying a method is called at *least* a certain number of times? In that instance, we can't fail the test until the actions of the object under test are finished -- there's no real way of catching the object misbehaving "in the act" like there is when setting an expectation for a maxumum number of calls. So, a spy/verify approach is perfectly reasonable.

At this point, some people may be worrying about consistency: if I use strict mocks only some of the time (during setup/arrange part of a test) and then spies some of the time, then it'll be 1) harder to read and 2) harder to understand when to do which style for people who are new to testing/mocks/etc. Both points are valid, and I therefore would recommend consistently doing strict mocks. Note that it is possible to go overboard with strict mocks, especially with parameter constraints or ordered calls, and end up with tests are are too tightly coupled to the implementation details. This can result in 1) tests that have a near-parallel change rate to the implementation, making for higher test maintenance; and, 2) cascading failures where many tests all fail for the same reason  -- both issues are classic ways of reducing the ROI for developer testing. Sometimes, this is a smell that the interface of the class(es) you are mocking has too granular of a public interface. Sometimes the tight coupling between tests and implementation details is someone from a design-by-contract (DBC) background not realizing the longer-term maintenance issues they might be introducing by introducing things into the test that do not relate to the specific scenario the test is documenting. It can also be someone who's just plain excited about the mock framework being used wants to try every feature of it. It's also possible to write spies that are tightly coupled to the implementation as well, so there's no panacea in either style for solving the problem of properly onboarding and mentoring new developers in your codebase and developer testing practices at large.

In summary, all Ruby and JavaScript (front-end and Node.js) developers should take half a day and learn to use the debugger in whatever their IDE/runtime of choice is. Add a throw/raise and make sure the debugger stops, inspect the different stack frames, and explore variables/objects in those stack frames. Consumers of test/mock frameowkrs, and the designers of those frameworks, need to be mindful of optimizing not just the writing and reading aspects, but also the debugging aspects when a given assertion or mock constraint fails. If the observe->understand->fix->re-test cycle is too expensive, then the longer-term maintenance cost/friction of a reasonably growing test suite will upset the ROI of developer testing and potentially cause a backlash. Like most things when it comes to development practices, the whys and the hows are fairly complex and nuanced -- there's no bumper sticker way to drum up zealotry. 

* I think a big part of why medium- to long-term developer productivity issues don't come into play is because often these frameworks are being driven by consultants who never have to deal with a given product/project beyond the first 3-6 months or first 6-12 full-time engineers. As a result, a lot of perceptions, practices, and frameworks put forth by shorter-term consultants tend to over-optimize for the front-end inception/bootstrapping of a project and its initial developers.

The first game we played on PlayStation 3

It was the spring of 2008 when my husband came home and said "I was just at Doug's house, and we played this game called something-something Eden that we have to own." "What platform was it on?", I asked. "I think it was a PlayStation 3, the controllers are different from our Xbox."

Up until that point, I had been putting off getting a then-next-gen console. We had bought an Xbox in 2002 after getting fed up with the never-ending troubleshooting of graphics and sound drivers involved in PC gaming. That Xbox was one of the best electronics purchases we had ever made: it supported the full resolution of our Panasonic 42" widescreen television, most of its games made effective use of surround sound, there were quite a few multiplayer games, and many of the PC games we liked or were anticipating were available on Xbox. All that value was even before we modded the Xbox -- XBMC opened up a whole new world of content from YouTube, 1up, and Revision3. As of 2007, we were pretty set for games (and media)*.

When Geoff came home and made the request for me to purchase a gaming machine to play PixelJunk Eden on, Toshiba has just dropped the whole HD-DVD format (other than in China) just a few weeks before. Still, the hardware failures of the Xbox 360 had put me off of all then-next-gen consoles until new hardware revisions would come out. (I tracked the Xbox 360 hardware reverse engineering community super closely, waiting for a 360 with a motherboard that didn't have fatal flaws.) I did a lot of research on the PlayStation 3 hardware and failure rates and became quite impressed with the capabilities of the architecture (then-crazy amounts of memory bandwidth), and the motion sensor technology/sample rates on the controller which put the Wii-mote to shame.  With Blu-ray having won as the HD video physical disc format and becoming convinced on the PS3's hardware viability I went to Fry's Electronics and bought a PS3 bundle that included Metal Gear Solid 4 (which we played for 30 minutes and then discarded) and an extra controller. I also bought Pan's Labyrinth on blu-ray, along with one or two other movies.

Our first 5 or 10 game purchases were on the fledgling PlayStation Store digital storefront. As I recall, the console even had a free PSN code for PAIN (still one of my mom's favorite games on her own PS3). PixelJunk Eden was one of the first, along with Echochrome, Everyday Shooter, and Blast Factor. I still vividly remember surprising Geoff on his birthday by coming home early, getting everything set up, and then hiding the PS3 controller underneath a pillow on the couch. When he got home, I asked him to help me "find my keys" to get him to discover the controller himself. He was really surprised, and we played for hours that night.

PixelJunk Eden, for all its surface simplicity in screenshots and verbal description, was simply amazing to behold at the time. First, because the PS3 supported HDMI output, our receiver was getting LPCM (uncompressed) 5.1 surround sound from the PS3. PixelJunk Eden didn't really make effective use of surround sound, but because audio wasn't being decoded from low-bitrate MP3 and then re-encoded as Dolby Digital, the dynamic range and clarity were an enormous leap from the original Xbox. (The incredibly unique synth sounds of the PixelJunk Eden soundtrack, especially in the first few minutes, are an aural experience I'll never forget.)  Second, the color palette was insanely vibrant and took great advantage of the wider color gamut afforded by the modern HDMI connector -- even though our decent Panasonic HDTV didn't have the best contrast ratio. Third, while some Xbox games supported widescreen 720p (The Warriors and X-Men Legends being two favs) , none of them rendered at 60 frames per second or had HD-native assets -- the smoothness and responsiveness was impossible to describe and just had to be experienced to understand. 

Seeing and hearing PixelJunk Eden in motion totally blew us away, and the fact it supported fluid and fun co-operative play out of the box sealed the deal. Even though the Xbox 360 had backward compatibility for many of our favorite XBox games we still owned (Fuzion Frenzy, Grabbed by the Ghoulies, Halo, etc), our eyes were opened to a totally different concept of what gaming could be: an immersive, beautiful experience that was even more fun to share together than by ourselves. We never looked back, totally skipping Xbox 360 as a platform, and still haven't regretted it.

Post-script: When we upgraded our TV to a Panasonic Plasma with a dramatically better dynamic contrast ratio and native 1080p resolution a few years later in 2010, we realized we were only experiencing about HALF of the background layers and other details on our previous HDTV. The improved resolution and contrast on the new plasma TV actually made a number of the trophies easier to get, and we played through the entire game and the Encore DLC a second time. It was just as fun and captivating as the first time. I'm looking forward to experiencing PixelJunk Eden again when our children are old enough to play video games with their dads.

* [I must say that I was *extremely* excited for the Nintendo Wii leading up to its release. I had friends at Ubisoft who had told me some details of the development kits and capabilities, and it sounded very cool and significantly more capable than the Gamecube. At some point, the later development kits had significantly scaled-down hardware with a totally different GPU and significantly reduced texture memory. From that point on, the Wii kept getting referred to as an "overclocked Gamecube" and both the quality of the motion sensors and the sample rate of the motion sensor data were cut in half. At that point, I tuned out of Nintendo Wii as a platform I would personally spend money on.]