Serverless Computing: AWS Lambda Function Invocation from Aurora MySQL – Second Part of a Journey

Today the journey of invoking a AWS Lambda function and storing its result into a table is completed. Some more issues had to be overcome.

Recap

The previous blog left off with the realization that in MySQL 5.6 there is no native facility to parse JSON objects as returned from the AWS Lambda invocation.

Recursive Descent Parser

To keep things simple, I implemented a recursive descent parser in MySQL (not 100% complete, only the part needed for this blog – still missing functionality to be implemented later as needed). The parser also extracts a property being sought for at the same time.

The signature is as follows:

CREATE PROCEDURE json_extract_property
  (IN  json_document TEXT,
   IN property_name VARCHAR(32),
   OUT property_value VARCHAR(32),
   OUT property_type VARCHAR(6))

Fundamentally, given a JSON document and a property name, return the property value and JSON type if the property exists. Otherwise the value is SQL NULL; this allows for testing if a value was found or not.

Any error encountered is raised as a SQL exception according to the MySQL approach. For example, if an incorrect character is found in the JSON document, the following exception is raised:

SET message = CONCAT('Not a JSON document or JSON array: found incorrect "', 
                     current_char, '" at position: ', next_char_index - 1);
SIGNAL SQLSTATE '45000'
SET MESSAGE_TEXT = message,
MYSQL_ERRNO = 1001;

Current_char contains the character in question at this point of parser execution, and next_char_index points at the next character to be examined.

MySQL Table for Fibonacci

The functionality is as follows. The client inserts a number (into a table) for which the Fibonacci number should be computed. The table is defined as follows:

CREATE TABLE fib.fibonacci (
    fib INT,
    value INT,
    code CHAR(5),
    msg TEXT
);

The table has four columns, one holding the number for which the Fibonacci number should be computed for, one holds the Fibonacci number, and two columns for error handling – one holding the error code, and one the error message.

The client inserts a row as follows:

INSERT INTO fib.fibonacci VALUES (5, null, null, null);

This then causes a trigger to be executed that computes the Fibonacci number via AWS Lambda invocation and inserts the resulting value (or error message). The trigger is explained next.

MySQL Trigger Computing Fibonacci

The trigger is defined as follows:

DELIMITER $$
CREATE TRIGGER fibcomp BEFORE INSERT
ON fibonacci
FOR EACH ROW
BEGIN
DECLARE function_result_message TEXT;
DECLARE output_value_as_string VARCHAR(32);
DECLARE property_type VARCHAR(6);
DECLARE code CHAR(5) DEFAULT '00000';
DECLARE msg TEXT;
DECLARE CONTINUE HANDLER FOR SQLEXCEPTION
        BEGIN
        GET DIAGNOSTICS CONDITION 1
        code = RETURNED_SQLSTATE, msg = MESSAGE_TEXT;
        SET NEW.code = code;
        SET NEW.msg = msg;
        END;

SET function_result_message = lambda_sync('arn:aws:lambda:us-west-2:<acct>:function:fibonacci',
                                          CONCAT('{"fib": ', NEW.fib, '}'));
CALL json_extract_property(function_result_message, 'output', output_value_as_string, property_type);
IF (output_value_as_string IS NULL)
THEN
    SET NEW.code = "-1";
    SET NEW.msg = 'No output value received';
ELSE
    SET NEW.value = output_value_as_string;
END IF;
END$$
DELIMITER ;

The trigger invokes the recursive descent parser. If a result is found, it is added to the row the user is inserting. If no value is found, code and error message are set. If there is any error being raised, this is recorded in the row as well.

After the above insert, the result is as follows:

| fib | value | code | msg  |
+-----+-------+------+------+
| 5   | 5     | null | null |

Issues Encountered

This second part of the journey was not without issues, either. Here a short summary:

The AWS RDS web user interface sometimes does not work. When I made a change to my MySQL instance, the change was not recognized by the UI and I could not apply the change.

The resolution to this is using the REST endpoints in order to modify the MySQL instance.

A bigger “issue” is that the function “lambda_sync” provided by AWS is formatting (pretty printing) the resulting JSON document before returning it adding ‘0A’ for pretty printing. It took me a while to realize that.

The resolution to this problem was to add the following to the parser as a first execution:

SET normalized_json_document = REPLACE(REPLACE(json_document, '\r', ''), '\n', '');

This ensures that any formatting characters added by lambda_sync are removed before parsing starts.

Note on Engineering with MySQL

In order to be able to efficiently work while implementing the recursive descent parser I installed MySQL locally not using the hosted version. This was very convenient and a route to consider.

Summary

In summary, I was able to implement the AWS Lambda function in such a way that a single function can be invoked from MySQL as well as via the AWS Gateway providing the same result structure.

In principle, the trigger mechanism allows the invocation of functions as well as their result processing – even though there is no support infrastructure from that provided by AWS. I assume this will get addressed over time by the appropriate upgrades of MySQL as well as making the lambda_sync function available.

Go Serverless!

Disclaimer

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

 

Advertisements

Serverless Computing: AWS Lambda Function Invocation from Aurora MySQL – First Part of a Journey

Lambda Invocation from Aurora MySQL: Setup

After creating an instance of Aurora MySQL it must get privileges assigned in order for it to be able to invoke AWS Lambda. Giving Aurora MySQL access to AWS Lambda is described in the following page. I followed that process to the last detail: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/ AuroraMySQL.Integrating.Lambda.html

Lambda: First Invocation from Aurora MySQL

A first invocation of my AWS Lambda function is implemented as simple SQL select statement – mostly for testing. It looks like the following:

SELECT lambda_sync('arn:aws:lambda:us-west-2:<...>:
                    function:fibonacci',
                   '{"fib": 5}') 
       AS 'result';

First Trouble: Input JSON Document Inconsistency

The return value of the select statement is a BLOB and it contains an error message:

{
   "body" : "{\"output\":\"Event body is null\",
              \"input\":\"{\\\"fib\\\":5}\"}",
   "isBase64Encoded" : false,
   "statusCode" : "400"
}

Now that is curious as the invocation with the exact same payload works when going through the API Gateway:

POST /test/ HTTP/1.1
Host: eflqvnlyoj.execute-api.us-west-2.amazonaws.com
content-type: application/json
Cache-Control: no-cache
Postman-Token: c5102b2c-bd89-4d89-af37-2fd46124a55a
{
"fib": 5
}

Returns

{
    "output": "5",
    "input": <...>
}

Error Analysis

The error message “Event body is null” is coming from my own function implementation. This error message is returned when the function cannot find a property called “fib” in the input payload. It looks like it finds “fib” when invoked through the API Gateway, but it does not find “fib” when invoked through Aurora MySQL. Why is that?

Based on the CloudWatch log the input parameter value (InputStream) to the Lambda function is different when coming from the API Gateway and the Aurora MySQL trigger. The value of the InputStream parameter of the HandleRequest method differs. The difference is as follows.

In case of the API Gateway the InputStream contains a JSON document consisting of the path, path parameters, headers, etc., providing the context of the REST invocation. The actual input payload is contained in a JSON top-level property called “body”.

In case of the invocation from Aurora MySQL the input is the JSON document as defined in the SQL select statement above (without further context like path, path parameters, etc. as it is in context of the API Gateway). Specifically, it is not wrapped within a JSON document with a property “body”.

Disappointing Inconsistency

First of all, this inconsistency is software-engineering-wise disappointing as a part of the function implementation now has to be conditional on the client invoking it. The payload from different clients is structured differently by AWS. Not good.

What should have happened is that the payload should be at the same location, no matter the invocation context or client. In addition, the context should have been qualified and separated out as it would have a different content depending on the client. For example, the InputStream value structure could have been like this:

{
“body”: <payload as sent>,
“context”: <a generic context object with a context type> 
}

The value of the context type could be the kind of possible invocation clients (environments) of functions like “API Gateway”, “Aurora MySQL 5.6”, etc., with details following outlining the specifics of the invoking client. This approach would have allowed a uniform function invocation as well as a uniform function implementation.

Error Resolution

There are different ways to deal with the inconsistent situation. One would be to refactor the function into three parts:

  • One function to be used when invoked from the API Gateway
  • One function to be used when invoked from the Aurora MySQL
  • One “core” function that implements the “raw” functionality and is reused by the previous two functions that are client specific

While this would be a possible approach, this would mean limited reuse of the client specific functions and a proliferation of the number of functions to manage within the AWS Lambda implementation and environment (in my example it would be three instead of one).

My idea, however, is to be able to use the same single function for both clients (in my case). This would be possible if there were a way to check from within the function implementation where the invocation is originating from. Or, if it is possible to probe the InputStream for indications about the invoking client.

Note, there is a context being passed into the function invocation as well (com.amazonaws.services.lambda.runtime.Context), however, this context does not provide a reliable way to distinguish the type of client. It is not fully populated, especially it does not contain a client type designation.

Therefore my chosen solution is

  • Probe for “body” (assuming that every API Gateway invocation adds this property to the InputStream)
  • If found then it must be a API Gateway call
  • If not found then it is a Aurora MySQL invocation

Clearly, this is not a general solution as functions can be invoked from many more types of clients, however, for the current implementation this approach is sufficient. In a complete implementation all possible clients of functions would have to be supported.

Modified Function Implementation

The function implementation now tests for “body”. If “body” is found (API Gateway), then this is extracted to be the “real” input to the function. If “body” is not found (Aurora MySQL), then the value of InputStream is taken as the “real” input.

So far, so good. Now the same function can be used for invocations from the Aurora MySQL trigger as well as an invocation from the API Gateway.

But wait!

More Trouble: Result Output Requirements

There is more trouble. For the API Gateway to work, the output has to contain specific JSON top level properties, and that output would not be the same response to the invocation from Aurora MySQL as it is API Gateway specific. Search for “api gateway function internal server error” and there is a variety of discussions on what has to be included.

So not only the input, but also the output is client specific. Second bummer!

The error resolution for this case (my goal) is to make the output the same for both clients by creating it accordingly. In context of the API Gateway and Aurora MySQL this is possible. The output is as the API Gateway requires it, and the Aurora MySQL trigger will have to process it as well (requiring logic to do so).

In my case a function output example is

{
    "isBase64Encoded": false,
    "body": "{\"output\":\"5\",
              \"input\":\"{<the whole input is contained here>}\",
              \"error\":null}",
    "statusCode": "200"
}

This results in the following when the function is invoked via the API Gateway:

{
    "output": "5",
    "input": "{<the whole input is contained here>}\"}",
    "error": null
}

However, at this point it is not clear to me if that would work for all possible AWS Lambda clients, or if AWS actually would force me to have different outputs for different clients.

Even More Trouble: Error Handling Behavior

The error handling behavior is different for API Gateway and the Aurora MySQL as well. If the payload has a JSON syntax error when invoking the function via the API Gateway, the API Gateway will not complain (neither will AWS Lambda) and the error handling is left to the function implementation itself.

When invoking the function via Aurora MySQL lambda_sync() and the payload has a JSON syntax error, the following error message is returned:

Error Code: 1873. Lambda API returned error: Invalid Request Content. 
Could not parse request body into json: Unexpected character (';' (code 59)): 
was expecting comma to separate Object entries  at [Source: [B@413d13c7; line: 1, column: 11]

The payload with the JSON syntax error is not passed on to the function, but caught before the function is being invoked. This means that AWS does some error handling itself before the function invocation and some reaction to the possible errors have to be implemented in the client code in context of Aurora MySQL in addition to the function implementation.

Trouble Keeps On Coming: MySQL JSON Function Support

Aurora MySQL version 5.6 has AWS Lambda invocation support (e.g. the lambda_sync() function), but no MySQL JSON function support. Aurora MySQL 5.7 has JSON function support, but not AWS Lambda invocation support. Go figure.

(As a side note, this is probably the reason why the AWS Lambda invocation examples from Aurora MySQL 5.6 do not show how a function result is processed in a synchronous function invocation).

Destination: Not Reached Yet

With all the trouble to go through and having had to work through the various inconsistencies in behavior, the blog did not reach its final destination to have the same function be used from both, the API Gateway and Aurora MySQL 5.6.

What is left to be done (as it is known to me so far) is functionality to interpret the returned JSON result in context of Aurora MySQL. I probably have to implement a MySQL function that takes the AWS Lambda function result JSON document and picks out the return value.

Once the return value is extracted, it can be further processed. The Aurora MySQL function invocation logic (in my case via MySQL database triggers) will described in the next blog as the second part of the journey.

Summary

In summary, reusing AWS Lambda functions by different AWS technologies that can act as function invocation clients should be straightforward and not require specific coding in the function implementation.

It is unclear why AWS did not ensure this as it would have been probably easier (at least from a user’s perspective) to add additional clients in the future without having to go through extensive testing and probably additional coding.

Another observation is that the various behaviors and constraints are not documented by AWS. The ecosystem is your friend in this context.

Go Serverless!

Disclaimer

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

Serverless Computing: MySQL as Function Trigger (Preparation)

In the previous blog an AWS Lambda function is invoked through the API Gateway by a client. How do I invoke (trigger) the same AWS Lambda function by a relational database? This blog sets up MySQL first as preparation, including a database trigger.

Create MySQL RDS Instance

I followed CHAP_GettingStarted.CreatingConnecting.MySQL.html to create an RDS (Relational Database Service) MySQL instance.

In order to connect to it from the MySQL Workbench you need to

  • find the connectivity information on the details page. Look for the “Connect” tile to find the connection endpoint and port
  • use the master username and password and try to connect. However, the connection must fail as no security group is created yet that allows access (I am correctly getting MySQL error 10060)
  • create a second security group and create a rule with custom TCP, port 3306 (in my case) and the IP address from which you are accessing the database (the AWS UI determines that automatically). The rule has to be assigned to the database instance via modification for it to take effect
  • alternatively you could create a rule that allows access by all traffic from anywhere. However, I prefer a more restrictive rule, even though I might have to update it when the IP address changes in the system from where I am accessing the MySQL instance

Now logging in from the MySQL Workbench on your laptop using the master user name and master user password is possible – that is going to make development for MySQL easier.

Create Aurora MySQL Instance

Turns out, creating a MySQL RDS Instance was a wasted effort for the goal that I set myself. It is not possible to call a AWS Lamda function from an MySQL RDS instance. Bummer. However, it is possible from Aurora MySQL.

So, I started over and created an Aurora MySQL. Once I had Aurora MySQL setup and running, I could continue with the MySQL user, table, trigger and function definition specification inside Aurora MySQL. I’ll refer to Aurora MySQL as MySQL for short in the following.

Setup MySQL User

For development I setup a user in the database instance as follows:

  • CREATE USER ‘cbmysqluser’@’%’ IDENTIFIED BY ‘cbmysqluser’;
  • GRANT CREATE ON fib.* TO ‘cbmysqluser’@’%’;

Additional commands are necessary during development to provide additional permissions. Here is the collection I ended up with over time. You might not have to use every single one of those, but anyway. Here is the list of grants

  • GRANT DROP ON fib.* TO ‘cbmysqluser’@’%’;
  • GRANT SELECT ON fib.* TO ‘cbmysqluser’@’%’;
  • GRANT INSERT ON fib.* TO ‘cbmysqluser’@’%’;
  • GRANT TRIGGER ON fib.* TO ‘cbmysqluser’@’%’;
  • GRANT CREATE ROUTINE ON fib.* TO ‘cbmysqluser’@’%’;
  • GRANT ALTER ROUTINE ON fib.* TO ‘cbmysqluser’@’%’;
  • GRANT EXECUTE ON fib.* TO ‘cbmysqluser’@’%’;
  • GRANT UPDATE ON fib.* TO ‘cbmysqluser’@’%’;
  • GRANT DELETE ON fib.* TO ‘cbmysqluser’@’%’;

Here some revocations

  • REVOKE CREATE ON *.* FROM ‘cbmysqluser’@’%’;
  • REVOKE ALL PRIVILEGES, GRANT OPTION FROM ‘cbmysqluser’@’%’;

And some management commands

  • SHOW GRANTS FOR ‘cbmysqluser’@’%’;
  • FLUSH PRIVILEGES;

Create MySQL Table

Having the user setup, now a table can be created by that user.

The idea is to use the table as the invocation interface. A value is inserted into a column (representing a function parameter), and after the function is executed the result is stored in another column into the same row. Each row therefore has the function parameter value as well as the corresponding return value.

In my case the function is the Fibonacci function. For example, calling it with 0 will return 0. Calling it with 10 will return with 55.

| fib | value |
+-----+-------+
|   0 |     0 |
|  10 |    55 |

Here the table creation statement

CREATE TABLE fibonacci (
  fib INT,
  value INT
);

Create MySQL Trigger

Now moving on to the trigger. I split the trigger into the trigger itself, and a separate function. The rigger, when fired, passes the input parameter to the function and stores the value computed by the function into the table. The trigger is specified as follows

DELIMITER $$
CREATE TRIGGER fibcomp 
  BEFORE INSERT ON fibonacci
  FOR EACH ROW
    BEGIN
      SET NEW.value = fibcomp(NEW.fib);
    END$$
  DELIMITER ;

The function computes the Fibonacci number as follows

DELIMITER $$
CREATE FUNCTION fibcomp (fib INT) 
  RETURNS INT 
BEGIN
  DECLARE f1 INT DEFAULT 0;
  DECLARE f2 INT DEFAULT 1;
  DECLARE sum INT;
  DECLARE i INT DEFAULT 2; 
  
  IF fib <= 0 THEN RETURN f1;
  ELSE
    WHILE i <= fib DO
      SET i = i + 1;
      SET sum = f1 + f2;
      SET f1 = f2;
      SET f2 = sum;
    END WHILE;
    RETURN f2;
  END IF;
END $$
DELIMITER ;

When issuing the following two insert statements, the above table content is the result

INSERT INTO fibonacci VALUES (0, null);
INSERT INTO fibonacci VALUES (10, null);

At this point I have the whole infrastructure available in context of MySQL and the functionality in place. This is the basis for incorporating the AWS Lambda implementation of the function implementing the Fibonacci computation next.

Summary

Once it was clear that Aurora MySQL is required, the setup of a database user, table, trigger and function was easy as this is done within MySQL not using any additional AWS infrastructure. With the ability to connect to the database instance using MySQL Workbench the development environment familiar to me was available, and that’s great.

Go Serverless!

Disclaimer

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.