Open array parameters and array of const

This article describes the syntax and use of open array parameters, and the use of the “array of const” parameter type. It also describes the internals of these two similar types of parameters, discusses lifetime issues, and gives a code solution for these issues. It has a short discusson on the confusion between open arrays and dynamic arrays, and between Variant arrays and variant open arrays.

Open array parameters

Open array parameters are special parameters which allow you to write procedures or functions (I will use the word routines, if I mean both) that can act on any array of the same base type, regardless of its size. To declare an open array parameter, you use a syntax like this:

procedure ListAllIntegers(const AnArray: array of Integer);
var
  I: Integer;
begin
  for I := Low(AnArray) to High(AnArray) do
    WriteLn('Integer at index ', I, ' is ', AnArray[I]);
end;

You can call this procedure with any one-dimensional array of Integers, so you can call it with an array[0..1] of Integer as well as with an array[42..937] of Integer, or even a dynamic type array of Integer.

The code also demonstrates how you can determine the size of the array in the routine. Delphi knows the pseudo-functions Low and High. They are not real functions, they are just syntactic items in Delphi that take the form of a function, but actually rely on the compiler to substitute them for code. Low gives the lower bound of an array, and High the upper bound. You can also use Length, which returns the number of elements of the array.

But if you call the code with an array that is not zero-based, like for instance in the following (nonsense) example,

var
  NonZero: array[7..9] of Integer;
begin
  NonZero[7] := 17;
  NonZero[8] := 325;
  NonZero[9] := 11;
  ListAllIntegers(NonZero);
end.

you will see that the output is like this:

Integer at index 0 is 17
Integer at index 1 is 325
Integer at index 2 is 11

That is because inside the procedure or function, the array is always seen as a zero based array. So for an open array parameter, Low is always 0, and High is adjusted accordingly (note that this is not necessarily true for other uses of High and Low, i.e. not on open array parameters). For open arrays, Length is always High + 1.

Slice

If you don’t want to use an entire array, but only a part of it, you can do that using the Slice pseudo-function. It is only allowed where an open array parameter is declared. It is used in this fashion:

const
  Months: array[1..12] of Integer = (31, 28, 31, 30, 31, 30,
                                     31, 31, 30, 31, 30, 31);
begin
  ListAllIntegers(Slice(Months, 6));
end;

That will only display the first 6 values of the array, not all 12.

var
  N: Integer;
begin
  N := 18;
  ListAllIntegers(Slice(Months, N));
end;

Internals

But how does that work; how can the function know the size of the array? It is quite simple. An open array parameter is actually a combination of two parameters, a pointer, which contains the address of the start of the array, and an integer, which contains the High value, adjusted for a zero base. So in fact the real parameter list of the procedure is something like this:

procedure ListAllIntegers(const AnArray: Pointer; High: Integer);

Each time you pass an array to an open array parameter, the compiler, which knows the size of the array, will pass its address and its adjusted High value to the procedure or function. For arrays of a static size, like array[7..9] of Integer, it uses the declared size to pass the High value; for dynamic arrays, it compiles code to get the High value of the array at runtime.

Usually, you can pass open arrays as const parameters. Open array parameters that are not passed as const will entirely be copied into local storage of the routine. The array is simply passed by reference, but if it is not declared const, the hidden start code of the routine will allocate room on the stack and copy the entire array to that local storage, using the reference as source address. For large arrays, this can be very inefficient. So if you don’t need to modify items in the array locally, make the open array parameter const.

Open array constructors

Sometimes you don’t want to declare and fill an array just so you can use it with an open array parameter. Luckily, Delphi allows you to declare open arrays on the spot, using the so called open array constructor syntax, which uses [ and ] to define the array. The above example with the NonZero array could also have been written like this:

ListAllIntegers([17, 325, 11]);

Here, the array is defined on the spot as [17, 325, 11].

Lifetime

An array created by an open array constructor is only valid as long as the function or procedure runs and is discarded right afterward. That is one reason why you can’t pass such an array to a var open array parameter.

Internally, the open array constructor simply makes room on the stack (the local variable frame) and puts copies of the values there. In other words, it creates an ad hoc array on the stack. Then it calls the function passing a pointer to the first value on the stack and the High value of that array. After the function call, the stack room is reclaimed, so the constructed array does not exist anymore.

Managed (reference counted) types like strings, interfaces, etc. are copied raw to the stack array and not finalized after the call, i.e. no reference counting is done. This is equivalent to passing a const parameter.

Confusion

Although the syntax is unfortunately very similar, an open array parameter should not be confused with a Delphi dynamic array. A dynamic array is an array that is maintained by Delphi, and of which you can change the size using SetLength. It is declared like:

type
  TIntegerArray = array of Integer;

Unfortunately, this looks a lot like the syntax used for open array parameters. But they are not the same. An open array parameter will accept dynamic arrays like array of Month, but also static arrays like array[0..11] of Month. So in a function with an open array parameter, you can’t call SetLength on the parameter. If you really only want to pass dynamic arrays, you’ll have to declare them separately, and use the type name as parameter type.

type
  TMonthArray = array of Month;

procedure AllKinds(const Arr: array of Month);
procedure OnlyDyn(Arr: TMonthArray);

Procedure AllKinds will accept static arrays as well as dynamic arrays, so SetLength can’t be used, since static arrays can’t be reallocated. Procedure OnlyDyn will only accept dynamic arrays, so you can use SetLength here (this will however use a copy, and not change the original array; if you want to change the length of the original array, use var Arr: TMonthArray in the declaration).

function Sum(const Items: array[1..7] of Integer): Integer;
procedure MoveTo(Spot: record X, Y: Integer; end);

type
  TWeek = array[1..7] of Integer;
  TSpot = record
    X, Y: Integer;
  end;

function Sum(const Items: TWeek): Integer;
procedure MoveTo(Spot: TSpot);

var
  MyDynArray: array of Integer;
begin
  MyDynArray := [17, 325, 11];

ListAllIntegers([17, 325, 11]);

TArray<T>

In versions of Delphi that know generics, you will see the type TArray<T> being used. It is declared as

type
  ...
  TArray<T> = array of T;

This is a generic type, and only useful if T is actually replaced by a concrete type, for instance

procedure OnlyDyn(Arr: TArray<Month>);

The construct is being used more and more, because generic types can circumvent the general rule that type compatibility is not dependent on the form of the declaration, but on where the type is defined. So not all array of Month are assignment compatible with each other, although they have exactly the same form. But all TArray<Month> are assignment compatible, even if they are declared on the spot. This is an exception to the rule I explain in the text box above. TArray<T> can be used as parameter and even as return type, so you can have declarations like the following.

constructor Create(Limbs: TArray<TLimb>; Negative: Boolean);
function ToByteArray: TArray<Byte>;

Assembler

To use an open array from assembler, you must remember that an open array is in fact a combination of two parameters. The first parameter is a pointer to the start of the array, the second the adjusted High value. Generally, open arrays are passed as const or var. In all cases, the array is passed by reference.

Here is a simple example of a function that sums all integers in an array:

function Sum(const Data: array of Integer): Integer;
// EAX: address of the array
// EDX: adjusted High value
asm
        MOV     ECX,EAX                 // P := PInteger(Addr(Data));
        XOR     EAX,EAX                 // Result := 0;
        OR      EDX,EDX
        JS      @@Exit                  // if High(Data) < 0 then Exit;
@@Next:                                 // repeat
        ADD     EAX,[ECX]               //   Result := Result + P^;
        ADD     ECX,TYPE Integer        //   Inc(PInteger(P));
        DEC     EDX                     //   Dec(EDX);
        JNS     @@Next                  // until EDX < 0;
@@Exit:
end;

The example above uses the usual register calling convention. If your function or procedure has a different calling convention, you may have to address the open array differently, but still as a combination of an address and a High value. If the procedure or function is in fact an instance method, then the first parameter (here: EAX) will be its implicit Self parameter, and Data will be passed in the next two parameters, in this case EDX and ECX.

Array of const

Array of const is a special case of open arrays. Instead of passing only one type, you can pass a variety of types. Have a look at the declaration of the Format function in Delphi:

function Format(const Format: string; const Args: array of const): string;

(Actually, there is a second, overloaded function in some versions of Delphi, but I’ll simply ignore that here.)

The first parameter is a string which indicates how you want your values formatted, and the second is an array of const, so you can pass a range of values using a similar syntax as the one for open array constructors. So you can call it like:

var
  Res: string;
  Int: Integer;
  Dub: Double;
  Str: string;
begin
  Int := Random(1000);
  Dub := Random * 1000;
  Str := 'Teletubbies';
  Res := Format('%4d %8.3f %s', [Int, Dub, Str]);
end;

Note: The official name for array of const parameters is variant open array parameters. This should not be confused with the Variant type in Delphi, and the arrays it can contain. They are quite different, even though a TVarRec (see below) is a bit similar to how a Variant is internally stored. Even the name of the internal record of a Variant is confusingly similar: TVarData.

Internals

Internally, an array of const is an open array of TVarRec. The declaration of TVarRec is given in the online help for Delphi. It is a variant record (also not to be confused with the Variant type), that contains a field called VType, and an overlay of different types, some of which are only pointers. The compiler creates a TVarRec for each member in the open array constructor, fills the VType field with the type of the member, and places the value, or a pointer to it, in one of the other fields. Then it passes the array of TVarRec to the function.

Since each TVarRec contains type information, Format can use this to check if it matches with the type given in the format string. That is why you get a runtime error when passing a wrong type. You can tell the compiler that you want it to store a different type identifier, by casting to the desired type. If the type doesn’t match one of the allowed types in a TVarRec exactly, the compiler will try to convert it to a matching type, so if you pass a Double, it will convert it to an Extended and pass that instead. Of course there are limitations on what the compiler can do, so for instance passing an object isn’t going to work.

Inside the function or procedure, you can treat the members as TVarRec immediately. The help for Delphi gives an example how to do this.

Lifetime issues

What you should notice is, that values in the TVarRec which are passed as pointers only exist during the course of the function or procedure. As soon as the routine ends, these values don’t exist anymore. So you should not be tempted to return these pointers from the procedure or function, or to store the TVarRecs in an array outside the routine, unless you can make sure that you manage the values yourself.

If you must copy the TVarRecs to an array or variable outside the function (this can also be a var parameter), be sure to make a copy (i.e. on the heap) of the value, and replace the pointer in the TVarRec with one to the copy. You should also take care that the copy is disposed of when it is not needed anymore. An example follows:

Download

type
  TConstArray = array of TVarRec;

// Copies a TVarRec and its contents. If the content is referenced
// the value will be copied to a new location and the reference
// updated.
function CopyVarRec(const Item: TVarRec): TVarRec;
var
  W: WideString;
begin
  // Copy entire TVarRec first
  Result := Item;
  // Now handle special cases
  case Item.VType of
    vtExtended:
      begin
        New(Result.VExtended);
        Result.VExtended^ := Item.VExtended^;
      end;
    vtString:
      begin
        // Improvement suggestion by Hallvard Vassbotn: only copy real length.
        GetMem(Result.VString, Length(Item.VString^) + 1);
        Result.VString^ := Item.VString^;
      end;
    vtPChar:
      Result.VPChar := StrNew(Item.VPChar);
    // There is no StrNew for PWideChar
    vtPWideChar:
      begin
        W := Item.VPWideChar;
        GetMem(Result.VPWideChar,
               (Length(W) + 1) * SizeOf(WideChar));
        Move(PWideChar(W)^, Result.VPWideChar^,
             (Length(W) + 1) * SizeOf(WideChar));
      end;
    // A little trickier: casting to AnsiString will ensure
    // reference counting is done properly.
    vtAnsiString:
      begin
        // nil out first, so no attempt to decrement reference count.
        Result.VAnsiString := nil;
        AnsiString(Result.VAnsiString) := AnsiString(Item.VAnsiString);
      end;
    vtCurrency:
      begin
        New(Result.VCurrency);
        Result.VCurrency^ := Item.VCurrency^;
      end;
    vtVariant:
      begin
        New(Result.VVariant);
        Result.VVariant^ := Item.VVariant^;
      end;
    // Casting ensures proper reference counting.
    vtInterface:
      begin
        Result.VInterface := nil;
        IInterface(Result.VInterface) := IInterface(Item.VInterface);
      end;
    // Casting ensures a proper copy is created.
    vtWideString:
      begin
        Result.VWideString := nil;
        WideString(Result.VWideString) := WideString(Item.VWideString);
      end;
    vtInt64:
      begin
        New(Result.VInt64);
        Result.VInt64^ := Item.VInt64^;
      end;
    vtUnicodeString:
      begin
        // Similar to AnsiString.
        Result.VUnicodeString := nil;
        UnicodeString(Result.VUnicodeString) := UnicodeString(Item.VUnicodeString);
      end;
    // VPointer and VObject don't have proper copy semantics so it
    // is impossible to write generic code that copies the contents
  end;
end;

function CreateConstArray(const Elements: array of const): TConstArray;
var
  I: Integer;
begin
  SetLength(Result, Length(Elements));
  for I := Low(Elements) to High(Elements) do
    Result[I] := CopyVarRec(Elements[I]);
end;

// use this function on copied TVarRecs only!
procedure FinalizeVarRec(var Item: TVarRec);
begin
  case Item.VType of
    vtExtended:
      Dispose(Item.VExtended);
    vtString:
      Dispose(Item.VString);
    vtPChar:
      StrDispose(Item.VPChar);
    vtPWideChar:
      FreeMem(Item.VPWideChar);
    vtAnsiString:
      AnsiString(Item.VAnsiString) := '';
    vtCurrency:
      Dispose(Item.VCurrency);
    vtVariant:
      Dispose(Item.VVariant);
    vtInterface:
      IInterface(Item.VInterface) := nil;
    vtWideString:
      WideString(Item.VWideString) := '';
    vtInt64:
      Dispose(Item.VInt64);
    vtUnicodeString:
      UnicodeString(Item.VUnicodeString) := '';
  end;
  Item.VInteger := 0;
end;

// A TConstArray contains TVarRecs that must be finalized. This function
// does that for all items in the array.
procedure FinalizeVarRecArray(var Arr: TConstArray);
var
  I: Integer;
begin
  for I := Low(Arr) to High(Arr) do
    FinalizeVarRec(Arr[I]);
  Arr := nil;
end;

The functions above can help you manage TVarRecs outside the routine for which they were constructed. You can even use a TConstArray where an open array is declared. The following little program

program VarRecTest;

{$APPTYPE CONSOLE}

uses
  SysUtils,
  VarRecUtils in 'VarRecUtils.pas';

var
  ConstArray: TConstArray;

begin
  ConstArray := CreateConstArray([1, 'Hello', 7.9, IntToStr(1234)]);
  try
    WriteLn(Format('%d --- %s --- %0.2f --- %s', ConstArray));
    Writeln(Format('%s --- %0.2f', Copy(ConstArray, 1, 2)));
  finally
    FinalizeConstArray(ConstArray);
  end;
  ReadLn;
end.

will produce the expected, but not very exciting, output

1 --- Hello --- 7.90 --- 1234
Hello --- 7.90

The little program above also demonstrates how you can use Copy to use only a part of the entire TConstArray. Copy will create a copy of the dynamic array, but not copy the contents, so you should not try to use Copy and then later on use FinalizeConstArray on that copy. In the program above, the copy will be removed automatically, since the lifetime of the copy is managed by the compiler.

Finally

Open arrays and arrays of const are powerful features of the language, but they come with a few caveats. I hope I succeeded in showing some of these, and how you can overcome them.

Questions regarding open arrays, array of const and many other language issues can be discussed on the Embarcadero Community forums (https://community.embarcadero.com/forum/programming), or e.g. on Stack Overflow.

Rudy Velthuis