Handily, vagrant supports ansible, so I can run a playbook from scratch simply by saying vagrant up . I had to smooth out some wrinkles: EC2 Ubuntu machine have an ‘ubuntu’ user, while Vagrant Ubuntu boxes have a ‘vagrant’ user. EC2 Ubuntu machines have a /dev/xvdb device, Vagrant boxes don’t.

And thus, this little shim playbook came to be:

- hosts: target

  user: vagrant

  sudo: True



  tasks:

    - name: Create fake block device for /dev/xvdb, simulating ephemeral storage

      action: file src=/dev/ram1 path=/dev/xvdb mode=0660 owner=root group=disk state=link



    - name: Create fake block device for /dev/xvdf, simulating an EBS volume

      action: file src=/dev/ram2 path=/dev/xvdf mode=0660 owner=root group=disk state=link



    # As yet there's no ansible module for making a file system [1]. We try, and if mkfs fails

    # we presume it's because there's already a file system on that device, and eat the

    # failure.

    #

    # [1] https://github.com/ansible/ansible/pull/2776

    - name: Put a filesystem on block devices to emulate EBS volumes

      action: shell mkfs -t ext4 /dev/$item ; true

      with_items:

        - xvdb

        - xvdf



    - name: Create the ubuntu user

      action: user name=ubuntu state=present groups=admin



    - name: Make the ubuntu user an sudoer

      action: lineinfile dest=/etc/sudoers regexp="^ubuntu" line="ubuntu ALL=(ALL) ALL" state=present



    - name: Make /home/ubuntu/.ssh/

      action: file path=/home/ubuntu/.ssh/ state=directory



    # This is not as lame as it looks. The real playbook will run as the 'ubuntu' user,

    # and this is a test environment.

    - name: Copy the ansible key from the vagrant user to the ubuntu user

      action: shell cp /home/vagrant/.ssh/authorized_keys /home/ubuntu/.ssh/authorized_keys creates=/home/ubuntu/.ssh/authorized_keys



    - name: Fix permissions of same

      action: file path=/home/ubuntu/.ssh/ owner=ubuntu group=ubuntu mode=0600 state=directory

      action: file path=/home/ubuntu/.ssh/authorized_keys owner=ubuntu group=ubuntu mode=0600 state=file



- include: therealplaybook.yml

A client of ours has an Enterprise Account and I was tasked with using their account to distribute the app for testing. To do this I needed to create a Distribution certificate (as opposed to a Development certificate). However, the option was greyed-out – disabled. What to do?

Eventually I tried Apple Support. They initially couldn’t figure out what was wrong either. They suggested that I didn’t have enough permissions (I did) or that the option was disabled because Enterprise Accounts can’t distribute through the app store – but they then confirmed that I needed that option to create an in-house app as well. After a few days I received the following email:

After discussing the issue with our specialists, I confirmed that there can only be two distribution certificates at one time. In order to create a new one, you would have to revoke one of the existing ones.

I presume that somewhere buried in the documentation this is mentioned, but certainly googling and my attempts to scan through the documentation from beginning to end didn’t turn anything up. So here for your googling needs:

Why is the option “App Store and Ad Hoc” disabled? It is because iOS Enterprise Developer accounts have a limit of two distribution certificates at a time.

An Emacs operator preparing for a discussion with a vim user about text editors.

For a concrete example, I’m working on a personal spaced repetition project, that also serves as an excuse to play with CQRS. The idea is that rather like Anki each card (fact in Anki terms) can have multiple fields, and questions and answers are templated.

In this example, I’ve split the view model repository into two by areas of functionality; one for editing cards (where we see N fields per card), and another for cards as they would appear whilst reviewing (were you just have a front and a back to the card).

However, there’s still some cruft here and there and dead code that we’d like to get rid of. Mutation testing makes it easier to find this, as deductively, if you mutate code that is never called, there should be no changes to the result of your program.

Running mutant over the new SRSRB::CardEditorProjection class, we (eventually) see the following in the final report:

!!! Mutant alive: rspec:SRSRB::CardEditorProjection#card_for:/Users/cez/projects/srs-rb/lib/srsrb/card_editor_projection.rb:60:235ac !!!
@@ -1,4 +1,4 @@
def card_for(id)
- cards.get[id]
+ cards.get
end
Took: (4.74s)
!!! Mutant alive: rspec:SRSRB::CardEditorProjection#card_for:/Users/cez/projects/srs-rb/lib/srsrb/card_editor_projection.rb:60:81c99 !!!
@@ -1,4 +1,4 @@
def card_for(id)
- cards.get[id]
+ cards[id]
end
Took: (4.75s)
!!! Mutant alive: rspec:SRSRB::CardEditorProjection#card_for:/Users/cez/projects/srs-rb/lib/srsrb/card_editor_projection.rb:60:f83c4 !!!
@@ -1,4 +1,4 @@
def card_for(id)
- cards.get[id]
+ id
end
Took: (4.71s)
!!! Mutant alive: rspec:SRSRB::CardEditorProjection#card_for:/Users/cez/projects/srs-rb/lib/srsrb/card_editor_projection.rb:60:9e7ae !!!
@@ -1,4 +1,4 @@
def card_for(id)
- cards.get[id]
+ nil
end
Took: (4.59s)
!!! Mutant alive: rspec:SRSRB::CardEditorProjection#card_for:/Users/cez/projects/srs-rb/lib/srsrb/card_editor_projection.rb:60:61efb !!!
@@ -1,4 +1,4 @@
-def card_for(id)
+def card_for(s8e207872a38234087817)
cards.get[id]
end
Took: (4.76s)
!!! Mutant alive: rspec:SRSRB::CardEditorProjection#card_for:/Users/cez/projects/srs-rb/lib/srsrb/card_editor_projection.rb:60:12b0a !!!
@@ -1,4 +1,4 @@
-def card_for(id)
+def card_for
cards.get[id]
end
Took: (4.64s)

So, mutant has found that for all the ways it knows how to mutate the method #card_for (renaming parameters, deleting code, &c) the result is that the tests still pass. However, the smoking gun is renaming input parametersーby renaming the method parameter without renaming their usages, it’s quite clear that this method never actually gets calledーotherwise the tests for that mutation would fail with a NameError. And by inspection, we can see it’s not referenced, so we can easily excise it.

Of course, the original purpose of mutation testing wasn’t just to find dead code; it’ll also show you where you have missing tests, or an edge case for a behavior has been missed. For example, it’s quite informative to run this once with just your unit tests, and then again including your end to end tests, as that will quite clearly highlight any short-cuts you might have (even unwittingly) taken in your unit tests.

There’s a fair amount of research into readability, at least in English. Flesh-Kincaid, SMOG (Simple Measure Of Gobbledygook), Coleman-Liau, Automated Readability Index are among the more well-known such measures. Essentially all these measures do the same thing: how long are your sentences, and how long are the words you use? Polysyllabic sesquipedalianism, let alone egregious hyperverbosity and prolixity, decreases readability. (CLI score: 34.14) Using short words makes things more readable. (CLI score: 8.5)

Flesch-Kincaid and SMOG both suffer from measuring syllables. “Suffer”, because syllable-counting in English is not trivial. However, Coleman-Liau and Automated Readability Index just count word length. More amenable to calculation.

Some languages permit comments to be explicitly tied to things: Clojure’s docstrings, Smalltalk’s class comments. Given that, let’s look at using Coleman-Liau on a Smalltalk package’s class comments. The Coleman-Liau index maps text into a real number that represents the approximate education level required to understand the text, according to the US education system. Thus, a score of 10 represents the reading ability of a Grade 10 student, 14 that of a second year undergraduate, and so on.

Now to do this properly we need to tokenise the text into words and sentences. In a production system we’d need to be careful: splitting the streams by periods is insufficient because otherwise “The U.S. Postal Service is slow.” would parse as three sentences. But in the interests of clarity, we’ll ignore that: we treat sentences as delimited by periods, and words by spaces.

| classes cli |

classes := (PackageInfo named: 'Kernel-Classes') classes.

cli := [:str | | words sentences l s |

    words := (str splitBy: ' ') collect:

        [:each | each withoutLeadingBlanks withoutTrailingBlanks].

    sentences := (str splitBy: '.') collect:

        [:each | each withoutLeadingBlanks withoutTrailingBlanks].



    "The original formulation is 0.0588L - 0.296S - 15.8 where

       * L is the average number of letters per 100 words and

       * S is the average number of sentences per 100 words.

      We fold the constant factor into the coefficients to make

      the important things clear:

       * l measures the average word length, while

       * s measures the (reciprocal of the) average sentence length."

    l := ((str select: #isAlphaNumeric) size) / (words size).

    s := ((sentences collect: [:each | (each splitBy: ' ') size]) average) / (words size).

    (5.88 * l) - (29.6 * s) - 15.8].

classes collect:

    [:cls | {cls name. cli value: cls instanceSide organization classComment asString}]



"=>   an OrderedCollection(

    #(#BasicClassOrganizer -45.400000000000006)

    #(#Behavior 8.568979591836733)

    #(#Categorizer 18.796997792494484)

    #(#Class 10.976666666666667)

    #(#ClassBuilder 10.886810551558753)

    #(#ClassCategoryReader -0.9633333333333312)

    #(#ClassCommentReader -45.400000000000006)

    #(#ClassDescription 13.691774891774887)

    #(#ClassOrganizer 7.881632653061221)

    #(#Metaclass 19.016567834681037))"

(Side note: usually we’d say cls comment . However, ClassDescription >> #comment returns a template encouraging the reader to fill in the blanks, in the event of there being a missing class comment. That would throw out our calculations, so we route around the helper and go directly to the source of the comments.)

But look at that first result: BasicClassOrganizer ’s class comment is apparently readable by someone not even born yet! Of course, that’s because that class has no comment. That’s handy information in itself, although we’d be better off filtering those classes out and treat them separately.

So what about languages other than English? I’ve seen work on a Japanese readability index and a Chinese one. Applying Coleman-Liau and ARI to agglutinative languages like isiXhosa would probably not work: such languages have longer words than English, and less words in a sentence: “Indoda iyambona umntwana” has a CLI of around 17.5, indicating a sentence well outside the grasp of an adult without extensive tertiary education. The English translation, “The man sees the child”, has a CLI of -0.5!

It appears that English has had by far the most research into readability. Or of course the internet hasn’t overcome its English bias yet.

Markov chains offer a neat trick for generating surrealist blog oeuvres. They work by figuring out the probability of one word appearing after another, given a suitable corpus of input material.

The meat of the algorithm is surprisingly simple. Given a sequence of tokens (words and punctuation characters), you build a mapping between tokens and the frequencies of the tokens that appear one step to their right.

To give it a whirl, clone the GitHub repo and make sure you have Leiningen installed. Here’s a fairly noddy example:

$ lein repl
...
user=> (use 'blog-o-matic.core)
nil
user=> (build-frequency-map ["the" "cat" "sat" "on" "the" "mat"])
{"on" {"the" 1}, "sat" {"on" 1}, "cat" {"sat" 1}, "the" {"mat" 1, "cat" 1}

Using the frequency map, you derive a new stream of tokens from a particular starting token and following the trail by making a weighted random choice among the available next tokens. The random-next-token function takes care of this. There’s also some helpers for stitching the tokens back together and spitting out sentences. Here’s a sample run based on the last twenty blog posts:

user=> (require '[blog-o-matic.scrape :as scrape])
nil
user=> (def posts (take 20 (scrape/fetch-posts)))
#user/posts
user=> (def posts-tokenised (map scrape/tokenise posts))
#user/posts-tokenised
user=> (def freqs (reduce build-frequency-map {} posts-tokenised))
#user/freqs
user=> (first (random-sentences freqs))
"With github-differ, I won’t regret it to serve as mentioned on BitBucket."
user=> (apply str (interpose " " (take 3 (random-sentences freqs))))
"Partly that stuff, and relatedly, by MongoDB on a sense therefore that spits
out the above. Android users won’t just inserted into the rub: The app wants
your submitter claims to click around to define your table and do. We’re using
a provisional response codes, and take a good work."

The devil is always in the details, and Markov chains are sensitive to the peculiarities of the input text. As is so often the case in the software world, 99% of the code is munging data into shape and 1% is a nifty algorithm. LShift blog posts present a challenge because you’ll find identifiers, magic numbers and great heaving code stanzas slap-bang in the middle of a sentence. Those bits have to be stripped out or the algorithm will get lost down a blind alley of symbology.

You may need to run it a few times before it produces anything giggle-worthy. Automated silliness detection is left as an exercise for the reader…

Our first local policy at LShift although we’ve had a Meteor is necessary to distinguish this point though, we want to and hence won’t discuss. Empty is the dark ages trying to wind position when your job to generate input data API calls to understand this blog post. Hello, add our socket buffer, etc.

I’m glad I have addressed your submitter claims to work out what if we just the two commits! You especially love peer review. Perhaps it’s about the motherboard. Success!

Some quick googling suggests there’s no permissions. Last month we leverage the language used between the bottom: Let’s mirror the same process.

The above the collections that empty dictionaries for escape from our model when you ever at which is primarily a matter of different outcomes sets of the random from a very much of the core operations on my Debian fan speed for new Android compatibility, sometimes parsers accept the working on Monday morning to encode being used by making it also like this product of Programming Languages as the news a programmer must either use spatial logic and quiet case being a method should work with the top level timer wheels.

WebDriver’s high-level library for remote service. Never Mind the value. Programming in. Unfortunately, since I did I wanted the parent right, let’s look at it. Entity Framework as well.

N-dimensional options as represented by receiving a Raspberry Pi! Monticello versions in. Imagine we’re at next time. Create button press.

YOUR CPU should have been researched for the Blog with the most convenient because we just being played with unnatural amounts of subscribers is the first ingredient we don’t care, We write some point, ignorant users and they’re in the drive.

HTML-formatted notification hooks compared to get the trunk model the wrapper functions or maybe I’m not bored or may find just starting point will all; try CorePy code is a structure produces. Objects remain.

RegCreateKey Operation is started before Jersey and are correctly, on a small program that method as well as JSON without cycles in a notification channel on Postgres, and look back asking for no harm to use packages together with Meteor Angular Leaderboard demo into an arbitrary number of the DoSomething message.

Network protocols live in the misfortune of all in most protocols often another word; after all of sense to rely on Spring Jersey Start Meteor in trait that fixes it with watch this code whenever Meteor data?

HEAD of the same answer: Aha!

The application uses Entity Framework as an ORM layer over an SQL database. One option would have been to use database triggers to log changes. However, we wanted to track changes not in terms of the database model, but in terms of the application model. This allows us to then easily query the log at a later point using the same application model and make sense of it. We can directly re-hydrate the logs back into C# objects in different recorded states in a strongly typed manner.

The solution is to override the database context object’s SaveChanges method. However, inside that statement is quite a lot of complexity! I have spent some time over the last few weeks extracting the logging code from the client application and the result is FrameLog, an open source logging library for Entity Framework.

It is compatible with EF version 5, and will work with both a code-first and database-first approach. The library in both binary and source forms can be used and modified for both commercial and non-commercial works (see the license).

Links for FrameLog:

• Source code, issue tracker, and documentation
• Binaries
• NuGet package

One of the features of FrameLog is the HistoryExplorer class, that allows type-safe querying of the logs. Here is an example:

Note that this example elides some User class that is used for recording who made the changes. Also note that you won’t just be able to drop in a FrameLog binary and do the above – there is a little work to be done setting up the database’s “FrameContext”. See the documentation.

But let’s jump straight to the chase and open the core dump file like in gdb like so:

OK, so far we’ve learned two things: firstly, the program died on SIGABRT (signal 6), which is raised by the abort(3) call, which is amongst other things invoked on failed asserts and secondly that there are no debug symbols in this java binary, hence no source level debugging which will make things more… interesting.

Some quick googling suggests there’s no quick and easy way to get a debug build for Oracle JDK (yet another reason to use OpenJDK…).

Undeterred we start out with a look at the b(ack)t(race):

Right, so it looks like java segfaulted whilst trying to read an entry from a zipfile. Let’s look at the relevant stackframe:

OK, at this point the absence of debug symbols starts to make itself painfully felt and I wink over my colleague Jarek who’s got l00ter GDB skills than myself. Deprived of some nice local variables or source code to look at we can still turn to disassemble the address the instruction pointer ( rip register) points to and look at things at the ASM level like so:

But let’s skip that for now and take a step back. The main thing I’d actually like to know is what zipfile java was trying to read when it segfaulted. But without debug symbols and thus the ability to inspect local variables and function call args it’s not obvious how to do so. The first ingredient we need to make progress is the relevant function signature.

Thankfully, Java is GPL’ed so we could download the source, but a bit of Googling is quicker, the signature we’re looking for is:

Another quick search tells out what jzfile looks like:

Fortuitously the filename is right at the beginning of the struct, which means we could get at it easily if we just had a pointer to the struct instance itself: we just need another pointer dereference (by contrast getting at the content of char* comment for example would be more involved, because we would not only need to work out the size of all the preceding entries in the struct but also how much padding the compiler added for alignment).

But how do we get at the pointer

that was passed as the first arg to

? Well, since I was running this on a 64-bit unix work station, we need to look up the linux calling conventions on amd64. It turns out the first argument it passed in the RDI register. So let’s have a look at the registers:

Right, so we want to inspect the content at 0x7ff3641a9d40 .

Alternatively, since we’re on a 64bit (= 8 bytes) machine, this gives the same answer:

Let’s do the second de-reference:

Aha! The segfault occured as java was trying to read in SOME_PROJECT’s resources jar!

Since gdb can also evaluate simple C expressions with the p(rint) command, we could also directly have done:

Success!

But what if we had indeed wanted to look at

instead? The easiest way would probably have been to do let the compiler do the work of computing the offset for us by writing a small program that defines the same struct type and prints out the offset between the two pointers. Or we could have just loaded a debug build of a program that uses that struct as a bogus symbol table in gdb. At this point though, I’m glad I only rarely need to leave the comfortable confines of a high level VM these days.

Without further ado, here it is: https://github.com/jaceklach/github-your-time