chamook.lol

An emacs macro to transform DTOs

Sun, 16 Feb 2025 15:30:01 +0200

I really like how emacs macros let me capture and replay my own stupid approaches to problems.

I had a bunch of DTO classes that looked like this:

public class ThingContainer
    {
        [JsonPropertyName("thing_one")]
        public string ThingOne { get; set; }
    
        [JsonPropertyName("thing_two")]
        public string ThingTwo { get; set; }
    }

and I wanted to convert them into records like this:

public sealed record ThingContainer(
        [property: JsonPropertyName("thing_one")] string ThingOne,
        [property: JsonPropertyName("thing_two")] string ThingTwo);

I'm sure Visual Studio or Rider has a feature for doing that, but I am just a simple country emacs user so I used some find and replace commands, and made a macro so it was easy to repeat. This is all using evil mode and some very vim commands.

First off, change class to sealed record:

:%s/class/sealed record

This is a change that was easy enough to apply to all the classes at once, so it doesn't need to be part of a macro.

Now to the fiddly bit, position the cursor on the opening curly boi and start a macro (C-x ( in my config) then do the following:

Select the whole body of the class

V %

Add the property prefix to the attributes:

:'<,'>s/JsonProperty/property: JsonProperty/

After each find/replace command the text is unselected so it needs to be selected again before running the next one. In this case, the cursor ends up on the last attribute, so moving down twice gets to the closing curly boi (j j) then selecting everything is the same as before:

V %

Remove public from all the properties:

:'<,'>s/public //

Select the whole class again by moving one line down j and then pressing V %

Remove the get and set, and the blank line as well:

:'<,'>s/ { get.*\n/,/

Select the whole class again by pressing V %

Remove the newline between the attributes and the properties:

:'<,'>s/]\n */] /

Move to the beginning of the class A to get to the end of the line % to jump to the opening curly boi

A to edit at the end of the line, then backspace twice to delete the curly boi and newline, replace it with (, esc to go back to normal mode after typing.

This change breaks the scope declaration so % won't work for getting to the end of the block anymore, instead use } to get to the next blank line, and b jumps backwards to the trailing comma. Select that and the closing curly boi with v e then c to replace it, type ); as the last part of our record, then esc for back to normal mode.

Finally C-x ) to finish recording the macro.

Now it can be used on all the other classes by moving the cursor to the right position and hitting C-x e

Doing it this way let me work through the changes I wanted to make one at a time, and then have a simple command for all the other classes where I wanted to make the change 😃

Stop Pretending To Be Stupid

Fri, 06 Oct 2023 09:00:01 +0200

There's a tone in much of the writing that I have read about software development lately, a tone that insists that we mustn't be too smart and we shouldn't do clever things in case the creations we produce from applying our intellect become too complex and thus too difficult to maintain.

I don't like it.

There are indeed merits to reducing complexity where possible - simpler things do tend to be well…simpler. However, I think this tendency to push away from complexity and away from doing something that is arbitrarily decreed as too smart is harmful both to the process of creating software and to the software that is produced.

Complexity is inherent to software. Software should aid us in performing real world tasks, those tasks have a level of complexity. So in order to manage those tasks, the law of requisite complexity asserts that the software must be at least as complex. It is not unreasonable to design software such that it can only manage a subset of operations within a task - but this is an explicit choice that should have ramifications across the software making it clear where it's limits lie, and providing necessary feedback and information to allow humans to take over.

This attitude of avoiding being clever extends to users as well. If we do not believe that the genius developer can handle complexity, our estimation of an end user is lower still. We do not expect that a lowly user could take over where the machine reaches its limit, and thus do not equip them to do so.

Stupidly Building

When we build software, we must consider what it is intended to do. A deeper understanding of the problem space allows us to create software that is correspondingly more able to provide assistance in solving that problem. Building software that is effective in a problem space also requires an understanding of the complexities in the software itself.

To reduce this overhead and to enable greater focus on the problem domain, we can learn development techniques that allow for more expressive code, or for more succinct handling of the required logic.

A quest for simplicity often stalls progress on both of these fronts. Problem spaces are not thoroughly explored, instead preferring to cleave to the happy path with the minimum amount of disruption from beginning to end. Software development is viewed in a similarly restricted fashion: more advanced techniques are never explored (despite their potential suitability for a problem space) and a lowest common denominator set of techniques is seen as an outer boundary.

The Happy Path

The happy path is a seductive proposition when starting on a new piece of software functionality. If we can focus just on what should happen when everything goes right then we can show speedy progress and everyone will be happy. But software has to operate in a world where everything doesn't always go as expected. Errors and outliers must be considered at some point, but continuing to work on them after showing a functioning system can easily be considered an unnecessary waste.

Avoiding errors at the start has more insidious ramifications than the obvious risk of the work to handle them being delayed. Defining a model in software that doesn't consider such outcomes can lead to a model that is incapable of handling them and difficult to modify such that it would be capable of dealing with them. If such a model becomes the source of truth for how a system should operate, it also hides any information about error states from any future developers. This causes a significant increase in the amount of work needed to even see the potential problems, never mind that needed to address them.

Using this hammer has really simplified all my problems

Code is the most malleable substance that we have available to work with. With it we can create and recreate systems of all sorts of shapes and sizes. This means that there are a great many tools and techniques that can be used in a many different programming paradigms from simple procedural scripts to total functional programming and everything else.

This wealth of knowledge ensures that for any given problem that software should help with, there is likely a technique that allows it to be represented clearly and worked with gracefully.

But in a quest for simplicity at all costs many tools and techniques are ignored or forbidden. We cannot adopt a new language/tool/framework/technique because we do not all understand it. This greatly limits the suitability of code that is produced to the problem at hand, and increases the chances that the complexities inherent in the problem will not properly be accommodated.

I could use a hammer to bang screws into place, but it would be more effective to use a screwdriver.

Stupid Software

Whether or not software complexity is considered during the building of software, it is inevitably encountered during its use. Software that is unsuited to the task for which it was intended harms those who rely on using it.

Even software that works as intended can still embody a lack of faith in any users that disempowers them, removing tools that would enable them to work more effectively in favour of more simplicity.

Your Name is Invalid

A frustrating flaw in many software systems is an encoding of rules about data that do not reflect the realities of that data. Software routinely tells users that their names are incorrect, their addresses cannot be found, their gender could only be one of two rigid options, or that they are performing a feat that is clearly impossible and could they just stop lying about it.

These deficiencies reflect a poor understanding of the data model on the part of the developers that seek to eschew the complexities of reality in favour of a simpler model. In many cases, they also show a missed opportunity for an even simpler solution - not collecting the data in the first place. But where data relating to humans is essential to the function of a system, it should be collected in a form that deals with the myriad complexities of human life. To do any less is to exclude the lived experiences of real people.

Dead Ends

When software is the only method by which a task can be accomplished, it must always be possible to accomplish the task using the software. A poor model of the task and the data involved can lead to dead ends, where no further steps can be taken because data collected excludes all options to continue.

It is not unreasonable to allow software to fail in complex tasks, but allowing that failing software to become a barrier to completion of the task is unacceptable. Software should be capable of the tasks for which it is intended, and where it fails it must provide an alternative path for a user.

Power(less) Users

Even where software is capable of performing in its intended role, it may still lack flexibility to offer users alternative ways of solving a problem.

Making software easy to use is a noble goal. The mantra that "you can't hold it wrong" is a good one. However someone approaches software, it should seek to be accommodating to them and guide them through performing the tasks that they need to complete with it. But people learn and grow in capabilities. What is suitable for a novice who must learn how to operate a system, is often limiting and slow for someone who has performed the same task many times before.

Refusing to offer more advanced ways of working with a system robs them of the potential to learn more about it. It operates as a sort of lock in that prevents them from learning more general skills that could be transferred from task to task.

In these cases, simplicity can be good until it is no longer appropriate. Humans are capable of handling complexities, and software ought to allow them to do so.

TL;DR

Software can be complex, but pretending that people are incapable of handling complexity leads us to shy away from dealing with it and that causes problems down the line. Instead, we must address that complexity head-on (even if that requires us to learn new things) in order to build software that is fit for purpose and beneficial to those who use it.

Business Value Isn't Real, Only Vibes Can Be Trusted

Mon, 28 Nov 2022 00:00:01 +0100

When Agile Influencers wade into attempting to answer the question of what to make rather than simply how to go about making it the concept of Business Value is often invoked as the measurestick by which to judge competing options. Simply pick the option that will deliver the maximum amount of Business Value and your project will surely be a success and your team will certainly be productive.

I do not believe this is correct. I believe that Business Value is an ill-defined idea, a lie, an irrelevance, and most assuredly not something you should allow to encroach on your decision making if you are serious about your craft (assuming of course, that misery is not your craft).

Blessed are the key stakeholders, for they will achieve the product goal

The Book of Scrum 4:19 (NIV)

What is valuable to a business anyways?

Under capitalism a business exists to concentrate wealth into the possession of its owners by making money ¹. This can be a motivating system to spur innovation and the production of goods and the deliverance of services that greatly improve the lives of all those who can purchase them. It can also lead to a bunch of weird dudes exploiting their way into hoarding a massive amount of wealth that they can use on meaningless days out and being dicks to people at websites. So I think we could say that at best it's a mixed bag.

It's cool and good that people suffer and die from preventable problems, because that means that the number representing my almost unimaginable wealth goes up

literally every billionaire

The obvious answer then to the question of Business Value is that anything that would make money for a business is valuable. However, even for things that directly deal with income or outgoings it is not always easy to directly identify which changes will lead to the business retaining a greater share of capital than it currently does - and this uncertainty is only amplified as the relationship becomes more indirect. This uncertainty is not limited only to a risk of developments not providing any return, but also the risk of any change actively harming the current state of affairs leading to losses.

Even if it were possible for a business to predict what actions would lead to the best returns (that is, those that return the greatest amount of profit) it is not certain that this aligns with the best outcome for anyone connected to the business who wants something other than maximum profit. Consider these possible actions:

a company fires many workers from its product team leading to savings on labour but the fired workers have now lost their employment, and the remaining team is left with more work to split among fewer members causing an increase in stess
adding extra data collection to an app allows a company to sell that user data to advertisers (or shadier organisations) which brings extra income for the company but worsens the user experience and doesn't fit with the ethics of the developers
rushing a new feature to meet market demands means that there is no time to properly evaluate the impact on the userbase and it's implementation leads to abuse

Each of these scenarios could include the business realising perceived business value, but the other negative effects make them the opposite of valuable to anyone with other values.

Business Value then is at best a vague uncertainty, and at worst actively harmful on a scale only limited to those things that the business can affect. Using this as a guide for what work should be done is not a responsible way to use people's labour.

A business without value?

Fine. Ok. A business has to make a profit or it can't exist. Maybe that's not a sensible way of organising our whole society given the nightmarish things we've seen it lead to. But fine, given the current arrangement of things we'll accept this for now².

But perhaps we can be a little bit more discerning than just blindly accepting the wild and unchecked pursuit of profit at all costs?

I did it all for the nookie

Fred Durst, proposing an alternative system of value

To simply remove the worst negative aspects of the pursuit of business value, doesn't solve the other challenges of determining what would offer positive returns - we need to find an approach that allows us to gain some insight into what does. For that matter, we also need to figure out a way to navigate with work that might not be quite so cartoonishly villianous, but could still have an impact that we aren't comfortable with.

How do we decide what's valuable?

We don't need to throw out all of agile at least, so we can start with working in small increments that we immediately make available to get fast feedback. But we are left needing a method of judging that feedback, and we also want to be able to judge our own work without needing to inflict it on our unsuspecting users. While the users may ultimately decide if we keep or revert a change, it is important that we can decide for ourselves which changes we want to focus on in the first place.

To this end, I think it is worth making a semi structured attempt to evaluate our own feelings about a potential change before, during, and after we make it. Because I am ageing and wish to cling to a semblance of my youth, I will call this a Vibe Check³.

Checking the Vibes

Search your feelings, you know this code is bad

Darth Vader (@l33td4rth on github)

I think it is important to start with a disclaimer that this is not merely intended to be a system of self gratification. We are bound by externalities, and the actions we take do affect others - so it is necessary to approach a process such as this with a clear understanding of what we aim to do and where the limits of our understanding may lie. If I am working on something that will impact people who have a significantly different life experience to my own it may fall outside of my own competence to evaluate the effects of my changes on those people - but as I do not wish to cause harm with my actions it would fall on me to learn about those impacts, this is best accomplished by having a person from that group as part of the team I work with so that they are constantly involved but if that is not available I can reach outside the team and use the good ol' fallback of talking with people.

Assuming that we are working as part of a team that has reached some level of consensus on what types of aims we have, and that we have methods in place to gain additional context from outside groups should we need it - we can proceed.

Thinkin' about the future

Before we make a change or build a thing, let's take a moment to consider what it is we're aiming to do and if the thing we're doing furthers that goal. Consider that there are many options of things to do at any given time, so if there is controversy about an option it could well be safe to just punt on it for now, and review it again later if some people still think it's a good idea - sometimes timing is a factor, but it's not worth compromising our integrity for a chance at good timing. Overall our goal should be to be generally happy and comfortable with what we're making, and happy and comfortable with the impact it has - but bear in mind that if something is difficult to do because we're doing it the right way to ensure it has a more positive impact, then that is good and the type of comfort we ought to be sacrificing rather than taking the easy way out but having worse outcomes for people affected by the system.

I promised (semi-)structure so here are some sample questions to ask at this stage, but it's not a prescriptive list so feel free to use your own questions that I am sure are definitely better than the ones I came up with and you should totally feel good about them.

Will this change let our users do more with the system?
Are we clarifying concepts in our domain with this change?
If we had this feature already, what kind of problems would it be causing for us?
Is there a simpler way to get the same benefit?
Does making this change prevent us from doing other things in the future?

Note that these questions are a mix of things that relate to the product and user experience, and things that relate to implementation details. I think it is important that we explicitly consider both at each stage of this process but understand that it will be necessary to make trade-offs. Naturally we want to be thrilled about both aspects, but sometimes providing a feature that is beneficial to users will require implementation work that is frustrating or boring and conversely sometimes engineering a solution properly will limit features that can be made available. The important thing is to be aware of both of these aspects and ensure that we are not violating our own constraints on either side for the sake of the other. Or in other words, don't do evil things because the tech was cool, and don't refuse to do good things because the tech is boring.

Conciously doing

As work progresses on any given feature/change/iteration we will learn more about what it is we're making. Hopefully, any new information we discover will simply confirm that we made a fantastic choice to work on this particular thing and that will be that, but more likely we will be presented with questions and challenges about the implementation of what we're doing, and possibly we might even discover that give us pause about the impact of it.

We need to make space as we work to address this information when we discover it, because if it does have a significant impact on how we want to proceed with the work it will save us time, frustration, and effort to consider that impact as it is made rather than progressing along a bad course and making the costs of change higher. I am certain that by this point, you're brimming with ideas for questions to ask when you discover information as you build but I will include some samples here for completeness:

Could we deliver a smaller version of this change to avoid technical challenges while still getting some benefit?
Should we expand the scope of this change so that it can properly deliver the value we wanted?
Can we reasonably delay this change to implement it in a better way, or should we focus on delivering the change now and change the way we implement it later?
[dramatically] Have we underestimated the impact of this change and should abandon it for now until we know it won't cause problems?
[less dramatically] Did we discover that this change won't be able to solve the problem that we wanted it to? Is it still worth implementing?

These questions imply the possibility of significantly increasing the amount of work that we need to do, or significantly decreasing the amount of stuff we deliver⁴ and this is often something that needs to be coordinated with other people. This potential for disruption when an expected timeline has been set is something that makes a lot of people uncomfortable but if we are to give a proper amount of respect and consideration to new information then it is to a degree unavoidable. It is important then to avoid conflict with people who might rely on our work, that we are clear up front about things that can happen during the process. Try where possible, to work with people on shared goals rather than a specific change or feature - it is easier to let go of a feature when it becomes clear that it is not meeting your needs than when you expect it is the solution to your problems.

Spectating retroactively

Some things are never done. Whenever we deliver something we want to know how well it is actually received, how fit it turned out to be for its purpose, how it should impact our future changes, and if we learned anything from the process that should change the way we work or the questions we ask in the future.

Look upon my works, ye mighty, and tell me on a scale of 1 and 10 how likely you would be to recommend them to a friend

Ozymandias, King of NPS

Releasing something new is exciting, it can be all too easy to become attached to what you have made and lose focus on what it was supposed to do, be on your guard against such feelings - remember the reasons that you do things, not only what you have done.

When something is out in the world and in use, it can be tempting to search for simple metrics that fit nicely onto a graph to evaluate how something was received - the graph is going up, so everything is good! Some metrics are good and useful, for example we know how fast something should respond when people are working with it so we can track the response times of our own systems to see if they fall into line. Other metrics are less useful, if we really want to get a feel for how a change is working out in the world, we need to seek out qualitative information from people who actually use it - the ideal scenario being the opportunity to watch people use the system and then talk with them afterwards.

The questions that you ask to users should be different to those that you ask yourselves, the goal when talking to them is to discover how they use a system and why they did it in that particular way - this can often give insights into how it could better work with them to gain a desired result, but it can also provide things to consider about how you got here. The following are some examples of questions that we may want to ask ourselves:

Did the results match our expectations?
Was our understanding of the problem domain complete?
How much information did we reveal while making something, and does that information change our expectations for the future?
Is there anything we learned during this that we didn't immediately act upon, but now we should do something with it?

Asking questions at this stage is necessarily concerned with the past, and thus has some limited scope for what we can impact - but it also has the most concrete information about what actually happened that can serve to increase our confidence in future changes. Given a comparison between our initial expectations and our actual measured outcomes, we can see the impact of any of our earlier interventions and use discrepancies to guide any future changes we wish to make.

Rinse and Repeat

As described earlier, work on a system should proceed in small increments where possible such that we can pass through each of the stages above many times during the lifetime of a system. It is important both to maintain the process of asking questions as we go, guarding against the complacency that comes with repetition to ensure that we keep questioning what we are doing; and to evaluate the process itself to ensure that our ways of working do not guide us into producing things that go against our goals.

Like with learning another language, or trying to lift heavier weights this is a process that rewards consistent effort over a long time rather than a single burst of herculean exertion. As we produce more new things, and have more history to evaluate and draw lessons from we can improve our every step towards the future.

What was all this about?

I tried so hard, and got so far, but in the end I didn't hit my KPIs

Chester Bennington

You have significant productive capacity, but you can only use it on a limited number of things. Those things should not be decided merely by profit, they should be your way of making the world better according to your own values and ethics. Ok fine, you don't really have to make things better, sometimes getting by is enough - but making things worse as you go isn't fine and you are responsible for stopping that from happening. In order to keep track of what you're doing and the impact that it's having you should be prepared to question things along the way, before, during, and after doing something. You also need to be prepared to deal with the ramifications of acting on the information you gained from those questions.

You get to decide these things, and you can produce great stuff - listen to the vibes, don't chase blindly after business value.

Footnotes

I do realise that software development is undertaken in contexts other than that of for a business, but I can't write about everything. If you are in one of those other contexts, maybe this post is less relevant to you, or maybe it is a trove of brilliant insights - I guess the only way to find out is to read it. xoxoxo

Should we accept this forever though? Send your answer on the side of a bottle filled with burning gasoline to your nearest billionaire's compound.

Do not feel the need to inform me that this phrasing is outdated, I simply do not wish to know.

⁴

They also imply the opposite of those, but less work and more stuff done are nice things so let's just enjoy when that happens eh? 🍹

Using Indego Robot Mowers from Homey

Fri, 26 Aug 2022 00:00:01 +0100

My brother-in-law has a Bosch Indego robot mower that he wants to control via Homey. There was no app readily available, but there is an API available that has several open source clients. I helped him create some blocks of HomeyScript to call the API and integrate the mower into the rest of his smart home. The code here leans heavily on the work already done in the Java Controller Application and in the pyIndego Python Library, especially the documentation of the protocol.

Authentication

Most requests in the API require a contextId value, and to get one of those we first need to make a POST request to the /authenticate endpoint providing some details about the client as well as a Basic authentication token.

The authentication token is built by base64 encoding a string comprised of your username and password separated by a colon. This can be done conveniently in most programming languages, such as javascript:

return btoa("username@email.com:secret-password");

As is, this will output the following (but don't forget to use your actual username and password if you want to connect to the API for real):

    dXNlcm5hbWVAZW1haWwuY29tOnNlY3JldC1wYXNzd29yZA==

(Note that the above code sample won't actually work in HomeyScript, so you'll need to use a different approach if you want to generate the token as part of that)

With the token generated we can get a contextId from the /authenticate endpoint:

POST https://api.indego.iot.bosch-si.com/api/v1/authenticate
    Authorization: Basic dXNlcm5hbWVAZW1haWwuY29tOnNlY3JldC1wYXNzd29yZA==
    Content-Type: application/json
    
    {
        "accept_tc_id":"202012",
        "device": "",
        "os_type": "Android",
        "os_version": "4.0",
        "dvc_manuf": "unknown",
        "dvc_type": "unknown"
    }

The token generated previously is provided in the Authorization header, while we can use sample data to populate most of the fields in the request. It is worth noting that the accept_tc_id field value will likely need to be updated in the future if a new revision of the terms and conditions for the API are released.

HTTP/1.1 200 
    Content-Type: application/json
    
    {
      "contextId" : "'3f2a9e8c-93cb-402e-a200-e325859f3ffe",
      "userId" : "0a86dc31-7136-4009-9ef6-61ac4cab696e",
      "alm_sn" : "000000000"
    }

This provides us with the contextId that is needed to make other requests.

Before making other requests we can make this in a HomeyScript function:

// pull values from the flow editor
    const user = args[0];
    const pwd = args[1];
    
    // btoa isn't available :(
    const buffer = Buffer.from(user + ':' + pwd);
    const headerData = buffer.toString('base64');
    const authHeader = 'Basic ' + headerData;
    
    const authRequestBody = {
        accept_tc_id: "202012",
        device: "",
        os_type: "Android",
        os_version: "4.0",
        dvc_manuf: "unknown",
        dvc_type: "unknown"
    };
    
    const result = await fetch('https://api.indego.iot.bosch-si.com/api/v1/authenticate', {
        method: 'POST',
        body: JSON.stringify(authRequestBody),
        headers: {
            'Authorization': authHeader,
            'Content-Type': 'application/json'
        }
    });
    
    // fail for any error and return any error message we were given
    if (!result.ok) {
        throw new Error(result.statusText);
    }
    
    const body = await result.json();
    
    // return just the context id because we don't care about the other values
    return body.contextId;

The two const values for user and pwd should be provided from the flow editor, and this will output the contextId as a text value, that can then be passed to another function to do something.

Get Available Devices

If you already know the serial number for your mower, you can skip this step and just use that to work with it directly. If you don't know the serial number or you have multiple mowers that you want to work with, there is an API endpoint that will list all the available devices:

GET https://api.indego.iot.bosch-si.com/api/v1/alms/
    x-im-context-id: 3f2a9e8c-93cb-402e-a200-e325859f3ffe

Which will give a list containing the serial number and status code for the mowers connected to the account:

HTTP/1.1 200
    Content-Type: application/json
    
    [ {
      "alm_sn" : "000000000",
      "alm_status" : 258
    } ]

The serial number is then used to get more detailed information or to control the mower.

Get Information About The Mower

Now we have the contextId and the serial number of the mower we want to work with, we can make two different calls to get information about the mower.

State

First /state will give information about the current state of the mower, we need to include the serial number of the mower we want to get information about in the url and the context id is provided as a header:

GET https://api.indego.iot.bosch-si.com/api/v1/alms/{serial-number}/state
    x-im-context-id: 3f2a9e8c-93cb-402e-a200-e325859f3ffe

Which gives a response like this:

HTTP/1.1 200
    Content-Type: application/json
    
    {
      "state" : 258,
      "enabled" : true,
      "map_update_available" : true,
      "mowed" : 98,
      "mowmode" : 1,
      "xPos" : 12,
      "yPos" : 15,
      "runtime" : {
        "total" : {
          "operate" : 100000,
          "charge" : 30000
        },
        "session" : {
          "operate" : 2,
          "charge" : 0
        }
      },
      "mapsvgcache_ts" : 1582506399367,
      "svg_xPos" : 131,
      "svg_yPos" : 111,
      "config_change" : false,
      "mow_trig" : false
    }

The status code can be looked up in the following table that is a combination of data found in both the projects that I linked at the start of this post and some extra details that my brother-in-law figured out:

Status Code	Description
0	Reading Status
101	Docked
257	Charging
258	Docked
259	Docked - Software Update
260	Charging (Ran out of power)
261	Docked (Not 258 State)
262	Docked - Loading Map
263	Docked -Saving Map
266	Leaving Dock
512	Mowing
513	Mowing
514	Relocalising
515	Loading map
516	Learning lawn
517	Paused
518	Border cut
519	Idle in lawn
520	Mowing
521	Mowing
522	Mowing
523	Spot Mow
524	Mow without Docking Station
525	Mowing
768	Mowing
769	Returning to Dock
770	Returning to Dock
771	Returning to Dock - Battery low
772	Returning to dock - Calendar timeslot ended
773	Returning to dock - Battery temp range
774	Returning to dock
775	Returning to dock - Lawn complete
776	Returning to dock - Relocalising
1005	Mowing
1025	Diagnostic mode
1026	End of life
1027	Service Requesting Status
1038	Mower immobilized
1281	Software update
1537	Stuck
64513	Sleeping (Docked)
99999	Offline

Operating Data

And then /operatingData which can provide more detailed information for some properties, again including the serial number in the url and the context id as a header:

GET https://api.indego.iot.bosch-si.com/api/v1/alms/{serial number}/operatingData
    x-im-context-id: 3f2a9e8c-93cb-402e-a200-e325859f3ffe

In a response that looks like this:

HTTP/1.1 200 
    
    Content-Type: application/json
    
    
    {
      "runtime" : {
        "total" : {
          "operate" : 100000,
          "charge" : 35002
        },
        "session" : {
          "operate" : 0,
          "charge" : 0
        }
      },
      "battery" : {
        "voltage" : 7.0,
        "cycles" : 0,
        "discharge" : -0.1,
        "ambient_temp" : 23,
        "battery_temp" : 23,
        "percent" : 70
      },
      "garden" : {
        "id" : 1,
        "name" : 1,
        "signal_id" : 3,
        "size" : 157,
        "inner_bounds" : 0,
        "cuts" : 0,
        "runtime" : 100000,
        "charge" : 35002,
        "bumps" : 281,
        "stops" : 90,
        "last_mow" : 3,
        "map_cell_size" : 120
      },
      "hmiKeys" : 12019
    }

HomeyScript

Knowing how these requests and responses look, we can make useful HomeyScript functions so we could display the information somewhere or include it as part of a flow.

Get Status

Query the state endpoint and return the status converted to a human readable string:

// pull values from the flow editor
    const contextId = args[0];
    const serialNumber = args[1];
    
    // get the current state
    const result = await fetch('https://api.indego.iot.bosch-si.com/api/v1/alms/' + serialNumber + '/state', {
        method: 'GET',
        headers: { 'x-im-context-id': contextId }
    });
    
    if (!result.ok) {
        throw new Error(result.statusText);
    }
    
    const body = await result.json();
    
    // convert the status code to human readable text
    switch(body.state) {
        case 0: return "Reading status";
        case 257: return "Charging";
        case 258: return "Docked";
        case 259: return "Docked - Software update";
        case 260: return "Docked (Ran out of Power)";
        case 261: return "Docked (not 258 State)";
        case 262: return "Docked - Loading map";
        case 263: return "Docked - Saving map";
        case 266: return "Leaving dock";
        case 513: return "Mowing";
        case 514: return "Relocalising";
        case 515: return "Loading map";
        case 516: return "Learning lawn";
        case 517: return "Paused";
        case 518: return "Border cut";
        case 519: return "Idle in lawn";
        case 523: return "Spot Mow";
        case 524: return "Mow without Docking Station";
        case 769: return "Returning to Dock";
        case 770: return "Returning to Dock";
        case 771: return "Returning to Dock - Battery low";
        case 772: return "Returning to dock - Calendar timeslot ended";
        case 773: return "Returning to dock - Battery temp range";
        case 774: return "Returning to dock";
        case 775: return "Returning to dock - Lawn complete";
        case 776: return "Returning to dock - Relocalising";
        case 1005: return "Mowing";
        case 1025: return "Diagnostic mode";
        case 1026: return "End of life";
        case 1027: return "Service Requesting Status";
        case 1038: return "Mower immobilized";
        case 1281: return "Software update";
        case 1537: return "Stuck";
        case 64513: return "Sleeping (Docked)";
        case 99999: return "Offline";
        default: throw new Error("Unknown state" + body.state);
    }

Get Battery Percentage

Query the operating data and return only the battery percentage value, this can easily be modified to return other values instead:

// pull values from the flow editor
    const contextId = args[0];
    const serialNumber = args[1];
    
    // get operating data
    const result = await fetch('https://api.indego.iot.bosch-si.com/api/v1/alms/' + serialNumber + '/operatingData', {
        method: 'GET',
        headers: { 'x-im-context-id': contextId }
    });
    
    if (!result.ok) {
        throw new Error(result.statusText);
    }
    
    const body = await result.json();
    
    //return battery percentage
    return body.battery.percent;

Control the Mower

With code in place to authenticate with the API and retrieve information about the mower it is quite straightforward to control the mower. We just need to make a PUT request to the /state endpoint with the desired state command:

PUT https://api.indego.iot.bosch-si.com/api/v1/alms/{serial number}/state
    x-im-context-id: 3f2a9e8c-93cb-402e-a200-e325859f3ffe
    content-type: application/json
    
    {
      "state": "mow"
    }

We can issue a mow command to start the mower, and a returnToDock command to stop it and have it go back to the dock.

In HomeyScript this can be done like so:

// pull values from the flow editor
    const contextId = args[0];
    const serialNumber = args[1];
    
    // send the request to the api
    const body = {
        state: "mow"
    };
    
    const result = await fetch('https://api.indego.iot.bosch-si.com/api/v1/alms/' + serialNumber + '/state', {
        method: 'PUT',
        body: JSON.stringify(body),
        headers: {
            'x-im-context-id': contextId,
            'Content-Type': 'application/json'
        }
    });
    
    if (!result.ok) {
        throw new Error(result.statusText);
    }
    
    // this doesn't return a body, so as long as it didn't fail it should be good

Putting it together

The code snippets in this post can be added to HomeyScript cards in the flow editor, and you can link up your robot mower to anything else controlled by Homey.

The screenshot below shows examples of controlling the mower with a virtual device, and monitoring the battery percentage with a virtual sensor.

Happy Automating!

Literate Programming with Org Mode 🦄

Fri, 12 Aug 2022 00:00:01 +0100

Recently I've been writing bicep for provisioning Azure services, the intent was to create a reusable template that could provision different services for different projects. Having never worked with bicep before I wanted to ensure I wrote good documentation that I could refer back to later, and decided to try out Org Mode for its literate programming features to ensure that I kept the documentation in sync with the code.

Structured Code

Basic org syntax is a lot like markdown but using different special characters (/ for italics instead of _ etc) but a more significant difference is the approach of using structural blocks for source code. Markdown uses ``` to denote a block of code, and some processors (like github's) can take a language argument (e.g. ``` csharp) to provide syntax highlighting. Org uses comparitively clunky looking #+begin_src and #+end_src markers to surround code blocks, but it provides a much richer feature set for them.

Code blocks take a language parameter just like the one some markdown processors provide, but within emacs this provides access to features from the language mode that you would use when editing a file of that type like syntax highlighting and autocompletion. 😀

Execution

For some languages, it is also possible to execute the code directly inside the org file and have any output placed in a results block similar to Jupyter Notebooks.

A javascript block with this header line:

#+begin_src js :results output replace :wrap src json

And this javascript code:

fetch('https://jsonplaceholder.typicode.com/todos/2')
      .then(x => x.json())
      .then(x => console.log(x));

Executing that code block will output a results section that wraps the result in a json source block:

{
      userId: 1,
      id: 2,
      title: 'quis ut nam facilis et officia qui',
      completed: false
    }

But code in org files isn't limited to staying within the file, with org-babel-tangle you can create code files from your documentation.

Tangling

By adding :tangle and a filename to the start of a block, you can output code from an org documentation file into a file that only contains code, and can even be part of a whole coding project.

Using M-x org-babel-tangle from within emacs on a file containing a block like this:

    #+begin_src csharp :tangle SampleClass.cs
    namespace Chamook.Sample;
    
    public sealed class SampleClass
    {
        public string Name { get; }
    
        public void Greet() => Console.WriteLine($"Hello, {Name}");
    }
    
    #+end_src

Will output a file called SampleClass.cs with the code as contents:

namespace Chamook.Sample;
    
    public sealed class SampleClass
    {
        public string Name { get; }
    
        public void Greet() => Console.WriteLine($"Hello, {Name}");
    }

This can be broken up across several source blocks and tangled to a single file:

First declare a namespace:
    
    #+begin_src csharp :tangle SampleClass.cs
    namespace Chamook.Sample;
    #+end_src
    
    Then declare the class:
    #+begin_src csharp :tangle SampleClass.cs
    public sealed class SampleClass
    {
        public string Name { get; }
    
        public void Greet() => Console.WriteLine($"Hello, {Name}");
    }
    #+end_src

Will tangle to a single SampleClass.cs file containing code from all the source blocks that tangle to it in the order that they appear in the org file.

One org file can also tangle to multiple code files by specifying different filenames in code blocks:

Namespace for my C# file:
    
    #+begin_src csharp :tangle SampleClass.cs
    namespace Chamook.Sample;
    #+end_src
    
    Oh that reminds me of a funny javascript story:
    
    #+begin_src js :tangle hahaha.js
    function pleaseLaugh() {
      return "hahaha";
    }
    #+end_src
    
    Then declare a class in the C# file:
    
    #+begin_src csharp :tangle SampleClass.cs
    public sealed class SampleClass
    {
        public string Name { get; }
    
        public void Greet() => Console.WriteLine($"Hello, {Name}");
    }
    #+end_src

Will output both the .cs and the .js files specified.

Conditional Tangling

For the templates I was creating from an org file, I wanted to be able to pick and choose which parts were included rather than just outputting everything. Conveniently emacs can evaluate an emacs-lisp expression as part of the :tangle definition on a code block, meaning that I could check a condition and either return a filename or "no" which disables tangling for that block. The check could look at variables set within the org file, but as I was working on a template that other people might want to use I made it check if some dotfiles existed in the same directory:

#+begin_src bicep :tangle (if (file-exists-p ".Config1") "infra/main.bicep" "no")

This block will only be tangled with the main.bicep file if there is a file called .Config1 in the same directory as the org file.

Backfilling Values with Noweb

Building templates that were configurable in this way led to a scenario where sometimes I would need to include values in an earlier part of a bicep file only if a later block was tangled. Source blocks in org mode support using noweb style markup to include either the contents or the result of evaluating other blocks in the document, which provides a nice way of solving this problem.

In the initial block that may need a value from later, I can enable noweb replacements by specifying :noweb yes and then add a reference to the later contents:

#+begin_src :noweb yes
    <>
    #+end_src

Then later include a named block, that performs the check for the dotfile and returns a value if it is present, or an empty string otherwise:

#name: maybe-from-later
    #+begin_src emacs-lisp :cache no
    (if (file-exists-p ".config2") "// the file exists" "")
    #+end_src

Tangling for the non-Emacs user

A template is less useful if it constrains people to use a specific editor (with a relatively steep learning curve) to be able to get any value from it, but fortunately Emacs can be called from the command line to tangle a file so I included a shell script with my template. (I actually tangled the shell script from the template itself 🤘):

#+begin_src bash :tangle tangle.sh :shebang "#!/bin/bash"
    emacs --batch \
          --eval "(require 'org)" \
          --eval "(setq org-confirm-babel-evaluate nil)" \
          --eval '(org-babel-tangle-file "file.org")'
    #+end_src

This passes several snippets of emacs-lisp for emacs to evaluate:

(require 'org)

First ensures that org-mode is loaded, then we edit the org-babel configuration slightly:

(setq org-confirm-babel-evaluate nil)

This disables the prompt before evaluating each code block so that a user doesn't have to type yes for each block that is evaluated. With that config change in place it's just a matter to call the function that will actually tangle our org file and tell it which file to operate on:

(org-babel-tangle-file "file.org")

With this the org file is tangled and nobody had to open an emacs window or learn any keybindings 😅

Exporting

After working with org files a bunch to create this template, I discovered that there is also a fairly customizable export functionality - by setting a few config values:

(setq org-html-doctype "html5")
    (setq org-html-postamble-format '(("en" "
    
      <- M-x find-more-content
    
    
")))
    (setq org-html-postamble 't)
    (setq org-html-preamble-format '(("en" "
    
      %t
      %s
    
    
      
        By Adam Guest - %d
      
    
    
")))

And adding some extra values to include in the html at the top of the document:

#+options: toc:nil num:nil html-style:nil html5-fancy:'t title:nil exports:both
    #+html_content_class: container
    #+html_head: 
    #+html_head: 
    #+html_head:
    #+html_head:
    #+html_head:
    #+html_head:
    #+html_head:
    #+html_head:https://chamook.lol/literate-programming-with-org/" />
    #+html_head:https://chamook.lol/literate-programming-with-org/card.png" />
    #+html_head:
    #+html_head:
    #+html_head:
    #+html_head:https://chamook.lol/literate-programming-with-org/card.png" />
    #+html_head:
    #+html_head:

I can use org-export-dispatch to generate this html page 😍