Monthly Archives: April 2017

Magic sprinkles for Capybara and PDF

One of the frequent things I end up doing these days is generating reports. When you create a start up called Report Hero, it comes with the territory. The two most common outputs that I’m generating are PDF, and word (yeah word, I’m sorry).

I’ve recently being going through some pain getting testing working with PDFs and capybara. It took me way to long to get to the answers, so I’m going to document what I ended up with.

First, it’s worth noting that at the time of writing, poltergeist doesn’t support downloading files:
https://github.com/teampoltergeist/poltergeist/issues/485
http://stackoverflow.com/questions/35585994/downloading-a-csv-with-capybara-poltergeist-phantomjs

This means that you’ll need to use a different driver, WebKit works well for me (https://github.com/thoughtbot/capybara-webkit). As mentioned on that github project page, by adding webkit you’ll probably need to make sure that you run your tests on CI with an xvfb server.

If you are on rails 3 (did I mention I was doing this work on a legacy project), the next thing to consider is that depending on your setup you’ll need to make it possible to have two threads running at the same time to help enable the wkhtmlwrapper gem (wicked pdf or pdfkit) to work. So in you’re test.rb environment you’ll need to set config.threadsafe!

Once you’ve got PDFs being downloaded, there a number of options for performing the tests around them. They essentially boil down to reading the content using something like PDF Reader, and then performing assertions on it.

Pivotal have a decent blog post talking about this: https://content.pivotal.io/blog/how-to-test-pdfs-with-capybara , and prawn have extracted a gem to help: https://github.com/prawnpdf/pdf-inspector . For what it’s worth in this project, I went with the prawn pdf-inspecter gem.

So, the final steps I ended up with are:

  • use the webkit driver so you can download the pdf (with appropriate CI settings)
  • (in rails 3) set config.threadsafe! so the pdf generation can happen
  • use pdf-inspector to extract content from the PDF and perform some assertions.

With this setup you can

Creating a Deliverable HTML Email on AWS Lambda with SES

Update: Prefer const over let

Creating deliverable rich html emails is a great goal for many web applications, communicating with your customers, and helping to send the messages that are beautiful and make your marketing/designers happy.

As discussed in this send grid article, there are quite a number of approaches for doing this, but one of the best options for ensuring that images in emails with images. The leaders today in 2017 are using cid and referencing attachments, and using data urls.

According to the comments in this Campaign Monitor blog post, the cid method is supported by all the major clients today. (ironically the post is talking about using the cooler approach of data urls for images).

With this background, the question is how to do this with AWS Lambda and SES.

Thankfully it’s really straight forward.

The simple steps are:

  • create a simple html email that references images using cid: as the protocol.
  • create a raw rfc822 email string that can be sent with the SES api.
  • use the ses.sendRawEmail method to send the email.

1 Create a simple html email

For example:

<html><body><p>Hello world</p><img src=”cid:world”></body></html>

Note that the source of the image is of the format cid:world, this cid will be what you specify when attaching the image to the blog post

2 Create a raw rfc822 email string

The mailcomposer  package a part of Nodemail(https://nodemailer.com/extras/mailcomposer/) provides a great simple easy to use api for creating rfc822 emails with attachments. When creating attachments you can specify cids to refer to them by, and you can specify the contents of the attachment with a local filename, a buffer, or even a http resource. It’s a great api.  Take a look at the npm page to see more. One example of using this package is:

const from = 'from@example.com';
const to = 'to@example.com';
const subject = 'Subject';
const htmlMessage = '<html><body><p>Hello world</p><img src="cid:world"></body></html>';
const mail = new MailComposer({
  from: from, to: to, subject: subject, html: htmlMessage,
  attachments: [{
    filename: 'hello-world.jpg',
    path: 'https://cdn.pixabay.com/photo/2015/10/23/10/55/business-man-1002781_960_720.jpg',
    cid: 'world'
  }]
});
mail.build(function(err, res) {console.log(res.toString())});

3 Send the email with SES

Take the buffer that you create and send it with SES.

const sesParams = {
  RawMessage: {
    Data: message
  },
};
ses.sendRawEmail(sesParams, function(err, res){console.log(err, res)});

Full example using promises

Let’s put it all together, and pull in some of the promise code that I talked about in an earlier blog post(http://www.rojotek.com/blog/2017/04/11/create-a-promise-wrapper-for-a-standand-node-callback-method/)

function createEmail(){
  const from = 'from@example.com';
  const to = 'to@example.com';
  const subject = 'Subject';
  const htmlMessage = '<html><body><p>Hello world</p><img src="cid:world"></body></html>';
  const mail = new MailComposer({
    from: from, to: to, subject: subject, html: htmlMessage,
    attachments: [{
      filename: 'hello-world.jpg',
      path: 'https://cdn.pixabay.com/photo/2015/10/23/10/55/business-man-1002781_960_720.jpg',
      cid: 'world'
    }]
  });

  return new Promise((resolve, reject) => {
    mail.build(function(err, res) {
      err ? reject(err) : resolve(res);
    });
  });
}
createEmail().then(message =>{
  const sesParams = {
    RawMessage: {
      Data: message
    },
  };
  return ses.sendRawEmail(sesParams).promise();
});

Creating emails that include attachments is really quite easy with node, lambda and ses. Doing this is a great step to delivering rich emails that look like what your designers want.

 

Create a Promise Wrapper For a Standand Node Callback Method

Update: Prefer const over let

JavaScript Promises are the future, and a great pattern for doing asynchronous javascript code (allegedly async await is an awesome way to do async javascript as well, but I’m not there yet). There are great APIs for working with promises, and many standard libraries for working with Promises.

Unfortunately not all libraries support promises. Fortunately it isn’t hard to wrap a standard javascript callback pattern api in a promisified version.

The Node.js way is to have callbacks with an error first callback. These are apis which are passed a callback function with the signature function(error, success);. For a good description see this decent blog post The Node.js Way – Understanding Error-First Callbacks.

The classic example they provide is read file:

fs.readFile('/foo.txt', function(error, data) {
  // TODO: Error Handling Still Needed!
  console.log(data);
});

To convert this to a promise, create a new promise object, which calls reject with the error, and resolve with the data. If this is wrapped in a function, you’ll end up with a nice promisified readFile as per the following:

const fs=require('fs');

function readFilePromise(fileName) {
  return new Promise(function(resolve, reject){
    fs.readFile(fileName, function(err, data){
      if (err) {
        reject(err);
      } else {
        resolve(data);
      }
    });
  });
}

For extra cool kid points, use arrow functions:

const fs=require('fs');

const readFilePromise = fileName => {
  new Promise((resolve, reject) => {
    fs.readFile(fileName, (err, data)=> {
      if (err) {
        reject(err);
      } else {
        resolve(data);
      }
    })
  })
}

or to shrink it a little bit more:

const fs=require('fs');

const readFilePromise = fileName => {
  return new Promise((resolve, reject) => {fs.readFile(fileName, (err, data)=> {err ? reject(err) : resolve(data)})})
}

or to go a bit crazy with the inlining, and make your javascript look almost like haskell 🙂

const fs=require('fs');

const readFilePromise = fileName => new Promise((res, rej) => fs.readFile(fileName, (e, d) => e ? rej(e) : res(d)));

So it’s easy to see that any asynchronous node callback style api can be wrapped in a promise api with 10 lines of readable code, or 1 line of terse javascript.

Adding rubocop to a legacy project

To add rubocop to a legacy project, first grab a .rubocop.yml that specifies your projects code style and then do:

rubocop -c .rubocop.yml --auto-gen-config --exclude-limit 500

Then you’ll want to include the automatically generated todo file into your .rubocop.yml.

inherit_from: .rubocop_todo.yml

Run rubocop. Any violations that you now see will be caused by your config overriding the todo exclusions.  Find the cops causing problems using rubocop -D

Then fix them by doing things like:

increasing your Metrics/LineLength in  the .rubocop.yml

or perhaps setting the Enabled to false.

By doing this you can relatively quickly add rubocop to a legacy project with settings matching an organisations coding style, ready for you to really start making a codebase better.