Somewhere in the overlap between software development, process improvement and psychology

Posts tagged “code

Building a custom lambda runtime for anything? Even Pascal? Yes! #lambda #reinvent #aws

At AWS Reinvent 2018 today, Werner Vogels said it was now possible to use any language in AWS Lambda. I thought I’d put that to the test!

I thought it’d be interesting to add lambda support for Pascal, specifically the FreePascal variant found with Lazarus (the free, cross-platform version of Delphi). Mainly because it doesn’t really fit and it’s a compiled language, but I do have a remaining soft spot for the Lazarus project so I thought I’d give it a go.

Fair warning, this doesn’t make lots of sense. As a compiled language, you can’t initialise things in the runtime and then call them from multiple instances of functions, it also doesn’t deliver an amazing cold-start experience since it needs compiling on each run.

But… it does work! I based this on an AWS tutorial for creating a custom bash runtime.

You can write a Pascal lambda function like this:

begin
  writeln('{"status":"200", "message":"hello from fpc lambda"}');
end.

To do this, save the above as function.pas and then create an execution role:

To create an execution role

  1. Open the roles page in the IAM console.
  2. Choose Create role.
  3. Create a role with the following properties.
    • Trusted entityLambda.
    • PermissionsAWSLambdaBasicExecutionRole.
    • Role namelambda-role.

    The AWSLambdaBasicExecutionRole policy has the permissions that the function needs to write logs to CloudWatch Logs.

Then we can simply create a lambda function using the command line like so:

zip function.zip ./function.pas
aws lambda create-function --function-name fpc-hello --zip-file fileb://function.zip --handler function.handler --runtime provided --layers arn:aws:lambda:eu-west-1:743697633610:layer:fpc-runtime:20 --role arn:aws:iam:::role/lambda-role

This works because I’ve made the custom runtime layer public so anyone can use it.

But that doesn’t really do much, what about parsing and returning JSON? Easy…

uses
  fpjson, jsonparser, sysutils;

var
  lambdaEventData: TJSONData;
  lambdaEvent: TJSONObject;
  nameParameter, outputJSON: string;
begin
  //get incoming JSON and parse
  lambdaEventData := GetJSON(ParamStr(1));

  // cast as TJSONObject to make access easier
  lambdaEvent := TJSONObject(lambdaEventData);

  nameParameter := lambdaEvent.Get('name');
  outputJSON := format('{"status":"200", "message":"hello %s from fpc lambda"}',[nameParameter]);
  WriteLn(outputJSON);

end.

You can then invoke the function simply from the command line or AWS console:

aws lambda invoke --function-name fpc-lambda-event --payload '{"name":"Mike"}' response.txt
cat response.txt

Selection_023

How it works

The custom runtime is a simple linux executable that receives events from aws and publishes results. That means that things that are compiled on Amazon Linux 2 are generally going to work.

When lambda runs, it loads your custom layers into the /opt directory, so modifying the fpc.cfg and paths to take account of that is all that’s needed. This isn’t an especially lean implementation of fpc, it includes most libraries and a good chunk of the fcl.

Here’s my bootstrap code:


#!/bin/sh
set -euo pipefail

PATH="/opt/fpc3/bin:${PATH}"
export PATH

PPC_CONFIG_PATH="/opt/"
export PPC_CONFIG_PATH
#env
# Processing
while true
do
HEADERS="$(mktemp)"
# Get an event
EVENT_DATA=$(curl -sS -LD "$HEADERS" -X GET "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next")
REQUEST_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)

#RESPONSE=$(./function 'EVENT_DATA')

RESPONSE=$(instantfpc --set-cache=/tmp/ ./function.pas "$EVENT_DATA")

echo "fpc response: $RESPONSE"

# Send the response
curl -s -X POST "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$REQUEST_ID/response" -d "$RESPONSE"
done

You can see the entirety of the code, including the runtime, FPC files and the 2 functions described above on github.

Advertisements

How to call a webservice api from Amazon Alexa using Javascript in a Lambda function

This wasn’t trivial and I’ve seen a lot of questions about it online so I thought I’d share how I did this to get Alexa to respond to questions about my kids pocket money 🙂

You can call a URL like this:
(note I’m assuming a JSON response, you’d need to modify for non-JSON)

var http = require('http');

function getWebRequest(url,doWebRequestCallBack) {
    http.get(url, function (res) {
        var webResponseString = '';
        //console.log('Status Code: ' + res.statusCode);

        if (res.statusCode != 200) {
            doWebRequestCallBack(new Error("Non 200 Response"));
        }

        res.on('data', function (data) {
            webResponseString += data;
        });

        res.on('end', function () {
            //console.log('Got some data: '+ webResponseString);
            var webResponseObject = JSON.parse(webResponseString);
            if (webResponseObject.error) {
                //console.log("Web error: " + webResponseObject.error.message);
                doWebRequestCallBack(new Error(webResponseObject.error.message));
            } else {
                //console.log("web success");
                doWebRequestCallBack(null, webResponseObject);
            }
        });
    }).on('error', function (e) {
        //console.log("Communications error: " + e.message);
        doWebRequestCallBack(new Error(e.message));
    });
}

You can then use it like this from inside an intent handler:

function getPocketMoney(intent, session, callback) {

    let shouldEndSession = false;
    let repromptText = null;
    let speechOutput = '';
    let cardTitle = '';

    let url = "http://mypocketmoneyservice/getpocketmoney?person=mike";

    getWebRequest(url, function webResonseCallback(err, data) {
        if (err) {
            speechOutput = "Sorry I couldn't connect to the server: " + err;
        } else {
            //something like this
            const balance = data.pocketmoney.person.balance;
            speechOutput = `${person} has £${balance} pocket money.`;
            callback({}, buildSpeechletResponse(cardTitle, speechOutput, repromptText, shouldEndSession));
        }
    });
}

Howto: Controlling LightwaveRF lights with a Raspberry Pi, Flirc and Logitech Harmony One IR remote

I’ve recently got into home automation and so thought it would be fun to integrate my smart lighting with my media centre and remote control. When I watch TV or streaming video I want the main lights to go off and the side/back lights to come on in my living room. When I pause I want the main lights to come up a little and when I play I want main lights to turn off again 🙂

My setup

I use LightwaveRF devices to control my main lights (2 dimmers) and various side-lights and screen backlights, I’ve also got a LightwaveRF Link hub which allows the Lightwave’d lights to be controlled by app, but also by simple UDP packets.

I’ve got a Logitech Harmony One remote control, because it allows you to setup custom sequences and controls my TV, DVR box and sound bar.

My solution:

I decided to use a Raspberry Pi 3 (now with Wifi and Bluetooth built in!) to do the lightwave udp packets and  Flirc to interpret the IR signals and convert them into simple text commands. Then I wrote a little python program that listens to the incoming commands from the Flirc USB Infrared receiver. There are other ways of interpreting the IR commands, but this was a super simple one!

Harmony One to Lightwave RF integration (more…)


Manipulating web content in Excel, or, My Excel can haz ajax?

Yes. Well, kinda. Not really. Er… maybe. Hey look, it works!

There’s not much better than Microsoft Excel for mucking about with big grids of stuff really easily. I’ve often used Excel to “screen scrape” or at least “DOM scrape” web content for various reasons. One of the most common is to get stuff from a web app that doesn’t provide a “download to csv” option or a decent API. Another reason is to grab stuff from an RSS feed to mix in to whatever my spreadsheet is doing, or just to use it as a REST client. So I offer you the following generic solution, I’ll base this one around the RSS example…

Wikipedia, the absolute source of all knowledge and truth, says that for something to be AJAX “the use of JavaScript and XML is not actually required, nor do the requests need to be asynchronous”. Despite this  “AJAX” is generally considered to be an initialism and acronym for “Asynchronous JavaScript and XML”. Taking both of these into consideration I must conclude that AJAX means whatever we want it to mean. For me it’s when I’m using an XMLHttpRequest object in one way or another to yank some xml/html/whatever from a web server.

In Excel I’m not using javascript asynchronously but I am using xml and even a XMLHttpRequest object via com interop so for me that qualifies as doing ajax in Excel.

How to do it:

1. Avoid late binding of com objects and help yourself out by adding a reference (code editor: Tools -> References…) from the Excel app to a library that will provide a XMLHttpRequest object. Happily, “Microsoft XML, v6.0” provides such a beastie for us.

2. Write some Ajaxy code to use the object like this:

Private Sub MMD_DoSomeAjaxyStuff()

'declare some useful vars
Dim req As XMLHTTP
Dim doc As DOMDocument
Dim url As String

'Some things to process the XML
Dim entries As IXMLDOMNodeList
Dim entry As IXMLDOmNode

'The RSS to get, naturally
uri = "https://mikemacd.wordpress.com/feed/"

'make a new XMLHttpRequest object
Set req = new XMLHTTP

'Load the xml
req.Open "GET", uri, , "", ""
req,Send
While req.ReadyState <> 4
  DoEvents 'yeah I know...
Wend

'Read the XML
Set doc = New DOMDocument
doc.loadXML req.responseText

'Do something with the RSS
Set entries = doc.getElementsByTagName("item")
For i = 0 To entries.Length - 1
    'Do something with the RSS like stick it in a cell
    Set entry = entries.Item(i)
    Cells(i + 1, 1).Value = entry.childNodes(1).Text
Next i

End Sub

3. w00t 😀