Group set of commands as atomic transactions (C++)

Question

We're designing the architecture of an embedded device (esp32). One of the tasks is that the device should connect to the internet and use a preprovisioned redeem code to register itself with our web cloud. Within the same message the device will upload a random generated password to the server which is then stored in an SQL database. This password will be used on every further startup of the device to login to the cloud. So it should also be saved into nonvolatile storage. Here's the set of operations in pseudocode:

if (password not found in internal memory)
{
  pw = create_new_password() (1)
  send_passsword_with_redeem_code_to_server(pw) (2)
  store_password_to_flash(pw) (3)
}

The problem with this is that operations (2) and (3) need to be carried out in an all-or-nothing fashion BUT both commands can also fail. If the storage to flash memory fails AFTER the database has registered the device password, the device would not be allowed to register a new password again and it would be floating. At the same time, if I were to store the password first to flash and then do the web request, the web request could fail OR the device could be powered of and the password would never reach the webserver, causing the device to "think" that it has registered on the next boot (as the password is found in flash) but it actually isn't and it would be floating again.

Are there techniques to group these two commands together as a set of atomic transactions which are either accomplished or discarded as a whole?

Answer 1

In practice, such atomic transactions do not exist. Instead, you can try to make all operations idempotent so that they can be safely retried.

Idempotence is particularly important for network operations. Normally, the device would send a query and receive a response. But networks are unreliable. Of course, sending the request might fail. But it can also happen that you send a request, it is successfully processed on the server, but then the network connection is lost and you never receive the response. Idempotence means that such operations can be retried safely, that it doesn't matter whether the request is issued 1 or 10 times. This is easier when you express those operations not as a state change or action, but as a desired state that you want to achieve.

Here, it would make sense to make that network request achieve the state “the device with this redeem-code is bound to this password”. This allows a new password to be set initially, and allows the device to verify that it is properly registered, but wouldn't allow the device to change the password later.

Then, this piece of code can safely create and store the password first, and then always retry the network request:

template<class T> T load_or_default(StorageLocation, T default);
template<class T> void store_or_fail(StorageLocation, T value);

// 1. ensure that we have created a password
auto password = load_or_default(PASSWORD, "");
if (!password) {
  password = generate_password();
  store_or_fail(PASSWORD, password);
}

// 2. ensure that we are registered
ensure_registration_with_server_or_fail(password, redeem_code);

As an optimization, you could store an additional variable that represents registration status:

// 2. ensure that we are registered
auto is_registered = load_or_default(IS_REGISTERED, false);
if (!is_registered) {
  ensure_registration_with_server_or_fail(password, redeem_code);
  store_or_fail(IS_REGISTERED, true);
}

The concept of idempotence is particularly important in the HTTP protocol. As analyzed by Roy Fielding, this protocol lends itself well to state transfer (REST APIs). In HTTP, certain methods like GET and PUT are assumed to be idempotent, whereas POST is not. Your registration could be expressed elegantly via a PUT-style operation.

Answer 2

To complete the change of the password, the device could login with the new password, and only since then the old password becomes invalid.

If the device logins with the old password instead, the device is passed through but the change of password procedure is understood as failed. At this point the change of password can be retried.

You also need two files for the password on the flash memory of the device. More recent is used assuming it is consistent (right size, maybe checksum). It can be deleted if found inconsistent, and then the older one is used. New password overwrites the older file.

For the short time the two passwords are valid but if they are complex enough I do not see this as a big security compromise.

Answer 3

Can you store the password to flash, then send the password to the server, then store a note to flash that the password has been uploaded? If the device sees a password on the flash but no note, then it should try to re-upload the password, then, if the uploaded succeeded or if it failed because it already has a password, store a note to flash.