Scoping in Go is built around the notion of code blocks. You can find several good explanations of how variable scoping work in Go on Google. I’d like to highlight one slightly unintuitive consequence of Go’s block scoping if you’re used to a language like Python, keeping in mind, this example does not break with Go’s notion of block scoping:

Let’s start with a common pattern in Python:

class Data(object):

    def __init__(self, val):
        self.val = val

    def __repr__(self):
        return('Data({})'.format(self.val))

li = [Data(2), Data(3), Data(5)]

print(li)

for d in li:
    d.val += 1

print(li)

Output:

The use of context in Go can help you pass metadata through your program with helpful, related information about a call. Let’s build an example where we set a context key, “stack”, which keeps a history of the function names called over the lifetime of the context. As we pass the context object through a few layers of functions, we’ll append the name of the function to the value of the context key "stack".

Go uses goroutines to execute multiple bits of code at the same time. Channels allow for the aggregation of the results of these concurrent calls after they have finished.

Consider a case where we want to make several GET requests to a server. The server takes some time to process each request, in many cases can handle many simultaneous connections. In a language like Python, we might do the following to make several requests:

Say we need a map to store various versions of a configuration in Go. Here is a simple example of the structure:

envs := map[string]string{
    "dev":  "1",
    "prod": "2",
}

Given this config map, we need to create an additional map that uses the same strings as the keys, but has functions for values. The catch is that the body of each function needs to make use of the value from its corresponding key. For example, functions["prod"] should have a value of type func() string and the body of that function should make use of the value, envs["prod"]. Here’s a concrete example:

Markdown is useful tool – these blog posts are written in it. I like Markdown because once you learn it, it feels invisible. It is minimal and intuitive. However, sometimes you need it to do things a little differently.

I ran into an issue where I had content which had to be written in only Markdown (no HTML) and later needed to be rendered as HTML and inserted onto a webpage, but I needed to add attributes to the HTML tags that were generated. The content itself needed to look like this:

Creating a Go module

We’re going to create a CLI tool for sending a message to a channel in Slack using the command line. This post is similar to my earlier post: Creating an Elixir Module. We’ll be using the chat.postMessage Slack API endpoint. Also, make sure you have a Slack API token.

Our CLI syntax will be:

$ ./slack -message 'hello world!' -channel @slackbot

First, make sure you have your $GOPATH set properly.

supervisor is a UNIX utility to managing and respawning long running Python processes to ensure they are always running. Or according to its website:

Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.

Installation

supervisor can be installed with pip

$ pip install supervisor

Given a script test_proc.py, start the process under supervisor as

$ sudo supervisorctl start test_proc

Now it will run forever and you can see the process running with

Querying S3 with Presto

This post assumes you have an AWS account and a Presto instance (standalone or cluster) running. We’ll use the Presto CLI to run the queries against the Yelp dataset. The dataset is a JSON dump of a subset of Yelp’s data for businesses, reviews, checkins, users and tips.

Configure Hive metastore

Configure the Hive metastore to point at our data in S3. We are using the docker container inmobi/docker-hive

Creating a Presto Cluster

I first came across Presto when researching data virtualization - the idea that all of your data can be integrated regardless of its format or storage location. One can use scripts or periodic jobs to mashup data or create regular reports from several independent sources. However, these methods don’t scale well, especially when the queries change frequently or the data is ingested in realtime. Presto allows one to query a variety of data sources using SQL and presents the data in a standard table format, where it can be manipulated and JOINed like traditional relational data.

Creating an Elixir module

To get a better handle on Elixir, I developed a simple CLI tool for sending files in Slack.

To create a new project, run

$ mix new slack_bot

This creates a new Elixir project which looks like this

├── README.md
├── config
│   └── config.exs
├── lib
│   └── slack_bot.ex
├── mix.exs
└── slack_bot
    ├── slack_bot_helper.exs
    └── slack_bot_test.exs

Navigate to the lib folder and create a folder inside it called slack_bot. We will use this folder for our code (slack_bot.exs) will remain empty. Then, create the files bot.ex and cli.ex. cli.ex will be the entry point to our application. bot.ex will contain the code for the specific Slack functions.