Applied Pokology

Back to blog... _____ ---' __\_______ ______)

Understanding Poke methods

__) __) ---._______) Jose E. Marchesi May 4, 2020 Poke struct types can be a bit daunting at first sight. You can find all sort of things inside them: from fields, variables and functions to constraint expressions, initialization expressions, labels, other type definitions, and methods. Struct methods can be particularly confusing for the novice poker. In particular, it is important to understand the difference between methods and regular functions defined inside struct types. This article will hopefully clear the confusion, and also will provide the reader with a better understanding on how poke works internally. The Packet ========== First we need to define some structure to use as an example. Let's say we are interesting in poking Packets, as defined by the Packet Specification 1.2 published by the Packet Foundation (none less). In a nutshell, each Packet starts with a byte whose value is always 0xab, followed by a byte that defines the size of the payload. A stream of bytes conforming the payload follows, themselves followed by another stream of the same number of bytes with "control" values. We could translate this description into the following Poke struct type definition: ,---- | type Packet = | struct | { | byte magic = 0xab; | byte size; | byte[size] payload; | byte[size] control; | }; `---- See the Poke manual for details on types, initialization values, constraint expressions etc. There are some details described the Packet Specification 1.2 that are not covered in this simple definition, but we will be attending to that later in this article. The process of building structs =============================== Given the definition of a struct type like Packet, there are only two ways to build a struct value in Poke. One is to map it from some IO space. This is achieved using the map operator: ,---- | (poke) Packet @ 12#B | Packet { | magic = 0xab, | size = 2, | payload = [0x12UB,0x30UB], | control = [0x1UB,0x1UB] | } `---- The expression above maps a Packet starting at offset 12 bytes, in the current IO space. See the Poke manual for more details on using the map operator. The second way to build a struct value is to _construct_ one, specifying the value to some, all or none of its fields. It looks like this: ,---- | (poke) Packet {size = 2, payload = [1UB,2UB]} | Packet { | magic = 0xab, | size = 2, | payload = [0x1UB,0x2UB], | control = [0x0UB,0x0UB] | } `---- In either case, building a struct involves to determine the value of all the fields of the struct, one by one. The order in which the struct fields are built is determined by the order of appearance of the fields in the type description. In our example, the value of magic is determined first, then `size', `payload' and finally `control'. This is the reason why we can refer to the values of previous fields when defining fields, such as in the size of the `payload' array above, but not the other way around: by the time `payload' is mapped or constructed, the value of `size', has already been mapped or constructed. What happens behind the curtains is that when poke finds the definition of a struct type, like Packet, it compiles two functions from it: a mapper function, and a constructor function. The mapper function gets as arguments the IO space and the offset from which to map the struct value, whereas the constructor function gets the template specifying the initial values for some, or all of the fields; reasonable default values (like zeroes) are used for fields for which no initial values have been specified. These functions, mapper and constructor, are invoked to create fresh values when a map operator @ or a struct constructor is used in a Poke program, or at the poke prompt. Variables in struct types ========================= Fields are not the only entity that can appear in the definition of a struct type. Suppose that after reading more carefully the Packet Specification 1.2 (that spans for several thousand of pages) we realize that the field `size' doesn't really stores the number of bytes of the payload and control arrays, like we thought initially. Or not exactly: the Packet Foundation says that if `size' has the special value 0xff, then the size is zero. We could of course do something like this: ,---- | type Packet = | struct | { | byte magic = 0xab; | byte size; | | byte[size == 0xff ? 0 : size] payload; | byte[size == 0xff ? 0 : size] control; | }; `---- However, we can avoid replicating code by using a variable instead: ,---- | type Packet = | struct | { | byte magic = 0xab; | byte size; | | var real_size = (size == 0xff ? 0 : size); | | byte[real_size] payload; | byte[real_size] control; | }; `---- Note how the variable can be used after it gets defined. In the underlying process of mapping or constructing the struct, the variable is incorporated into the lexical environment. Once defined, it can be used in constraint expressions, array sizes, etc. We will see more about this later. Incidentally, it is of course possible to use global variables as well. For example: ,---- | var packet_special = 0xff; | type Packet = | struct | { | byte magic = 0xab; | byte size; | | var real_size = (size == packet_special ? 0 : size); | | byte[real_size] payload; | byte[real_size] control; | }; `---- In this case, the global `packet_special' gets captured in the lexical environment of the struct type (in reality in the lexical environment of the implicitly created mapper and constructor functions) in a way that if you later modify `packet_special' the new value will be used when mapping/constructing _new_ values of type Packet. Which is really cool, but lets not get distracted from the main topic... :) Functions in struct types ========================= Further reading of the Packet Specification 1.2 reveals that each Packet has an additional `crc' field. The content of this field is derived from both the payload bytes and the control bytes. But this is no vulgar CRC we are talking about. On the contrary, it is a special function developed by the CRC Foundation in partnership with the Packet Foundation, called superCRC (patented, TM). Fortunately, the CRC Foundation distributes a pickle `', that provides a `calculate_crc' function with the following spec: ,---- | fun calculate_crc = (byte[] data, byte[] control) int: `---- So let's use the function like this in our type, after loading the supercrc pickle: ,---- | load supercrc; | | type Packet = | struct | { | byte magic = 0xab; | byte size; | | var real_size = (size == 0xff ? 0 : size); | | byte[real_size] payload; | byte[real_size] control; | | int crc = calculate_crc (payload, control); | }; `---- However, there is a caveat: it happens that the calculation of the CRC may involve arithmetic and division, so the CRC Foundation warns us that the `calculate_crc' function may raise E_div_by_zero. However, the Packet 1.2 Specification tells us that in these situations, the `crc' field of the packet should contain zero. If we used the type above, any exception raised by `calculate_crc' would be propagated by the mapper/constructor: ,---- | (poke) Packet @ 12#B | unhandled division by zero exception `---- A solution is to use a function that takes care of the extra needed logic, wrapping calculate_crc: ,---- | load supercrc; | | type Packet = | struct | { | byte magic = 0xab; | byte size; | | var real_size = (size == 0xff ? 0 : size); | | byte[real_size] payload; | byte[real_size] control; | | fun corrected_crc = int: | { | try return calculate_crc (payload, control); | catch if E_div_by_zero { return 0; } | } | | int crc = corrected_crc; | }; `---- Again, note how the function is accessible after its definition. Note as well how both fields and variables and other functions can be used in the function body. There is no difference to define variables and functions in struct types than to define them inside other functions or on the top-level environment: the same lexical rules apply. Methods ======= At this point you may be thinking something on the line of "hey, since variables and functions are also members of the struct, I should be able to access them the same way than fields, right?". So you will want to do: ,---- | (poke) var p = Packet @ 12#B | (poke) p.real_size | (poke) p.corrected_crc `---- But sorry, this won't work. To understand why, think about the struct building process we sketched above. The mapper and constructor functions are derived/compiled from the struct type. You can imagine them to have prototypes like: ,---- | Packet_mapper (IOspace, offset) -> Packet value | Packet_constructor (template) -> Packet value `---- You can also picture the fields, variables and functions in the struct type specification as being defined inside the bodies of Packet_mapper and Packet_constructor, as their contents get mapped/constructed. For example, let's see what the mapper does: ,---- | Packet_mapper: | | . Map a byte, put it in a local `magic'. | . Map a byte, put it in a local `size'. | . Calculate the real size, put it in a local `real_size'. | . Map an array of real_size bytes, put it in a local `payload'. | . Map an array of real_size bytes, put it in a local `control'. | . Compile a function, put it in a local `corrected_crc'. | . map a byte, call the function in the local `corrected_crc', | complain if the values are not the same, otherwise put the | mapped byte in a local `crc'. | . Build a struct value with the values from the locals `magic', | `size', `payload', `control' and `crc', and return it. `---- The pseudo-code for the constructor would be almost identical. Just replace "map a byte" with "construct a byte". So you see, both the values for the mapped fields and the values for the variables and functions defined inside the struct type end as locals of the mapping process, but only the values of the fields are actually put in the struct value that is returned in the last step. This is where methods come in the picture. A method looks very similar to a function, but it is not quite the same thing. Let me show you an example: ,---- | load supercrc; | | type Packet = | struct | { | byte magic = 0xab; | byte size; | | var real_size = (size == 0xff ? 0 : size); | | byte[real_size] payload; | byte[real_size] control; | | fun corrected_crc = int: | { | try return calculate_crc (payload, control); | catch if E_div_by_zero { return 0; } | } | | int crc = corrected_crc; | | method c_crc = int: | { | return corrected_crc; | } | }; `---- We have added a method `c_crc' to our Packet struct type, that just returns the corrected superCRC (patented, TM) of a packet. This can be invoked using dot-notation, once a Packet value is mapped/constructed: ,---- | (poke) var p = Packet @ 12#B | (poke) p.c_crc | 0xdeadbeef `---- Now, the important bit here is that the method returns the corrected crc _of a Packet_. That's it, it actually operates on a Packet value. This Packet value gets implicitly passed as an argument whenever a method is invoked. We can visualize this with the following "pseudo Poke": ,---- | method c_crc = (Packet SELF) int: | { | return SELF.corrected_crc; | } `---- Fortunately, poke takes care to recognize when you are referring to fields of this implicit struct value, and does The Right Thing(TM) for you. This includes calling other methods: ,---- | method foo = void: { ... } | method bar = void: | { | [...] | foo; | } `---- The corresponding "pseudo-poke" being: ,---- | method bar = (Packet SELF) void: | { | [...] | (); | } `---- It is also possible to define methods that modify the contents of struct fields, no problem: ,---- | var packet_special = 0xff; | | type Packet = | struct | { | byte magic = 0xab; | byte size; | [...] | | method set_size = (byte s) void: | { | if (s == 0) | size = packet_special; | else | size = s; | } | }; `---- This is what is commonly known as a "setter". Note, incidentally, how a method can also use regular variables. The Poke compiler knows when to generate a store in a normal variable such as `packet_special', and when to generate a set to a SELF field. A few restrictions ================== Given the different nature of the variables, functions and methods, there are a couple of restrictions: - Functions can't set fields defined in the struct type. This will be rejected by the compiler: ,---- | type Foo = | struct | { | int field; | fun wrong = void: { field = 10; } | }; `---- Remember the construction/mapping process. When a function accesses a field of the struct type like in the example above, it is not doing one of these pseudo `SELF.field = 10'. Instead, it is simply updating the value of the local created in this step in Foo_mapper: ,---- | Foo_mapper: | | . Map an int, put it in a local `field'. | . [...] `---- Setting that local would impact the mapping of the subsequent fields if they refer to `field' (for example, in their constraint expression) but it wouldn't actually alter the value of the field `field' in the struct value that is created and returned from the mapper! This is very confusing, so we just disallow this with a compiler error "invalid assignment to struct field", for your own sanity 8-) - Methods can't be used in field constraint expressions, nor in variables or functions defined in a struct type. How could they be? The field constraint expressions, the initialization expressions of variables, and the functions defined in struct types are all executed as part of the mapper/constructor and, at that time, there is no struct value yet to pass to the method. If you try to do this, the compiler will greet you with an "invalid reference to struct method" message. Happy poking! :)