«« Site Rescued »»

While the site is still going to move from its current host, a new site will now take its place. More Info.

The new site may have less content, but the core will now remain. And it will now play nicely with phones! Keep an eye on the DelphiDabbler Blog for news.

How to dynamically add data to an executable file (part 4 of 5)

Random payload access

Random payload access

The "Delphi way" of providing random access to data is to derive a class from TStream and to override its abstract methods – and this is what we will do here. Our new class will be called TPayloadStream. It will detect payload data and provide read write random access to it.

Not only does this approach provide random access but it also has the added advantage of hiding the details of how the payload is implemented from the user of the class. All the user sees is the familiar TStream interface while all the gory details are hidden in TPayloadStream's implementation.

Listing 9 shows the definition of the new class, along with an enumeration – TPayloadOpenMode – that is used to determine whether a payload stream object is to read or write the payload data. Note that in addition to overriding TStream's abstract methods, TPayloadStream also overrides the virtual SetSize method to enable the user to change the size of the payload. This is necessary because, by default, SetSize does nothing.

  TPayloadOpenMode = (
    pomRead,    // read mode
    pomWrite    // write (append) mode

  TPayloadStream = class(TStream)
    fMode: TPayloadOpenMode;  // stream open mode
    fOldFileMode: Integer;    // preserves old file mode
    fFile: File;              // handle to exec file
    fDataStart: Integer;      // start of payload data in file
    fDataSize: Integer;       // size of payload
    constructor Create(const FileName: string;
      const Mode: TPayloadOpenMode);
      // opens payload of file in given open mode
    destructor Destroy; override;
      // close file, updating data in write mode
    function Seek(Offset: LongInt;
      Origin: Word): LongInt; override;
      // moves to specified position in payload
    procedure SetSize(NewSize: LongInt); override;
      // sets size of payload in write mode only
    function Read(var Buffer;
      Count: LongInt): LongInt; override;
      // Reads count bytes from payload
    function Write(const Buffer;
      Count: LongInt): LongInt; override;
      // Writes count bytes to payload in write mode only

The public methods of TPayloadStream are:

  • Create – Creates a TPayloadStream and opens the named file either in read or write mode.
  • Destroy – Updates the payload footer, closes the file and destroys the object.
  • Seek – Moves the stream's pointer to the specified position in the payload data, ensuring that the pointer remains within the payload.
  • SetSize – Sets the size of the payload in write mode only. Raises an exception when used in read mode. Note that setting the size to zero will remove the payload and the associated footer record.
  • Read – Attempts to read a specified number of bytes into a buffer. If there is insufficient data in the payload just the remaining bytes are read.
  • Write – Writes a specified number of bytes from a buffer to the payload, extending the payload if required. Works only in write mode – an exception is raised in read mode.

The class also uses the following private fields:

  • fMode – Records whether the stream is open for reading or writing.
  • fOldFileMode – Preserves the current Pascal file mode.
  • fFile – Pascal file descriptor that records the details of an open file.
  • fDataStart – Offset of the start of payload data from the start of the executable file.
  • fDataSize – Size of payload data.

We begin our review of the class's implementation by examining Listing 10 which shows the constructor and destructor. Once again we are using classic Pascal un-typed files to perform the underlying physical access to the executable file. However this could easily be changed to use some other file access techniques.

constructor TPayloadStream.Create(const FileName: string;
  const Mode: TPayloadOpenMode);
  Footer: TPayloadFooter; // footer record for payload data
  inherited Create;
  // Open file, saving current mode
  fMode := Mode;
  fOldFileMode := FileMode;
  AssignFile(fFile, FileName);
  case fMode of
    pomRead: FileMode := 0;
    pomWrite: FileMode := 2;
  Reset(fFile, 1);
  // Check for existing payload
  if ReadFooter(fFile, Footer) then
    // We have payload: record start and size of data
    fDataStart := Footer.ExeSize;
    fDataSize := Footer.DataSize;
    // There is no existing payload: start is end of file
    fDataStart := FileSize(fFile);
    fDataSize := 0;
  // Set required file position per mode
  case fMode of
    pomRead: System.Seek(fFile, fDataStart);
    pomWrite: System.Seek(fFile, fDataStart + fDataSize);

destructor TPayloadStream.Destroy;
  Footer: TPayloadFooter; // payload footer record
  if fMode = pomWrite then
    // We're in write mode: we need to update footer
    if fDataSize > 0 then
      // We have payload, so need a footer record
      Footer.ExeSize := fDataStart;
      Footer.DataSize := fDataSize;
      System.Seek(fFile, fDataStart + fDataSize);
      BlockWrite(fFile, Footer, SizeOf(Footer));
      // No payload => no footer
      System.Seek(fFile, fDataStart);
  // Close file and restore old file mode
  FileMode := fOldFileMode;

In the constructor the first thing we do is to record the open mode and then open the underlying file in the required mode. Next we try to read a payload footer record, using the ReadFooter function we developed in Listing 3.

Setting fDataStart
fDataStart is the same as the size of the executable file because payloads always start immediately after the executable code.

If we have a footer we a get the start of the payload data (fDataStart) and the size of the payload (fDataSize) from the footer's ExeSize and DataSize fields respectively. If there is no footer record we have no payload so we set fDataStart to refer to just beyond the end of the file and set fDataSize to zero.

Finally the constructor sets the file pointer according to the file mode – in read mode we set it to the start of the payload while in write mode we set it to the end.

In the destructor we proceed differently according to whether we a are in read mode or write mode:

In read mode all there is to do is to close the file and restore the previous Pascal file mode.

In write mode we first check if we actually have a payload (fDataSize > 0). If so we create a footer record using the InitFooter routine we defined in Listing 2 and record the payload size and start position in the record. We then seek to the end of the new payload data, truncate any data that falls beyond its end (as will be the case when the data size has shrunk), then write the footer. If there is no payload data we truncate the file at the end of the executable code. Finally we close the file and restore the file mode.

That completes the discussion of the class constructor and destructor. Let us now consider how we override the abstract Seek, Read and Write methods. Listing 11 has the details:

function TPayloadStream.Seek(Offset: Integer;
  Origin: Word): LongInt;
  // Calculate position in payload after move
  // (this is result value)
  case Origin of
      // Moving from start of stream: ignore -ve offsets
      if Offset >= 0 then Result := Offset
      else Result := 0;
      // Moving from end of stream: ignore +ve offsets
      if Offset <= 0 then Result := fDataSize + Offset
      else Result := fDataSize;
    else  // soFromCurrent and other values
      // Moving from current position
      Result := FilePos(fFile) - fDataStart + Offset;
  // Result must be within payload: make sure it is
  if Result < 0 then Result := 0;
  if Result > fDataSize then Result := fDataSize;
  // Perform actual seek in underlying file
  System.Seek(fFile, fDataStart + Result);

function TPayloadStream.Read(var Buffer;
  Count: Integer): LongInt;
  BytesRead: Integer;   // number of bytes read
  AvailBytes: Integer;  // number of bytes left in stream
  // Work out how many bytes we can read
  AvailBytes := fDataSize - Position;
  if AvailBytes < Count then
    Count := AvailBytes;
  // Read data from file and return bytes read
  BlockRead(fFile, Buffer, Count, BytesRead);
  Result := BytesRead;

function TPayloadStream.Write(const Buffer;
  Count: Integer): LongInt;
  BytesWritten: Integer;  // number of bytes written
  Pos: Integer;           // position in stream
  // Check in write mode
  if fMode <> pomWrite then
    raise EPayloadStream.Create(
      'TPayloadStream can''t write in read mode.');
  // Write the data, recording bytes read
  BlockWrite(fFile, Buffer, Count, BytesWritten);
  Result := BytesWritten;
  // Check if stream has grown
  Pos := FilePos(fFile);
  if Pos - fDataStart > fDataSize then
    fDataSize := Pos - fDataStart;

Seek is the most complicated of the three methods. This is because the FilePos and Seek routines (from the System unit) that we use to get and set the file pointer operate on the whole file, while our stream positions must be relative to the start of the payload data. We must also ensure that the file pointer cannot be set outside the payload data. The case statement contains the code that calculates the required offset within the payload, depending on the seek origin. The two lines following the case statement constrain the offset within the payload data. Finally we perform the actual seek operation on the underlying file, offset from the start of the payload data. The method returns the new offset relative to the payload.

The Read method must ensure that the read falls wholly within the payload data. We can't assume that all the remaining bytes in the stream can be read, because the payload may be followed by a footer record that is not part of the data. Therefore we calculate the number of available bytes by subtracting the current position in the payload from the size of the payload data. If there is insufficient data to meet the request the number of bytes to be read is reduced to the number of available bytes.

Note that Read uses TStream's Position property to get the current position in the payload data. This property calls the Seek method which, as we have seen, ensures that the position returned falls within the payload data.

Write is quite simple – we just check we are in write mode and output the data to the underlying file at the current position if so. The number of bytes written is returned. The only complication is that we must check if the write operation took us beyond the end of the current data and record the new data size if so. Should the stream be in read mode Write raises an exception.

All that remains to do now is to override the SetSize method. Listing 12 provides the implementation.

procedure TPayloadStream.SetSize(NewSize: Integer);
  Pos: Integer; // current position in stream
  // Check for write mode
  if fMode <> pomWrite then
    raise EPayloadStream.Create(
      'TPayloadStream can''t change size in read mode.');
  // Update size, adjusting position if required
  if NewSize < fDataSize then
    Pos := Position;
    fDataSize := NewSize;
    if Pos > fDataSize then
      Position := fDataSize;

Obviously, we can't change the stream size in read mode, so we raise an exception in this case. In write mode we only record the new size if it is less than the current payload size. In this case we must also check if the current stream position falls beyond the end of the reduced payload and move the position to the end of the truncated data if so. The Position property is used to get and set the stream position. As noted earlier, this property calls our overridden Seek method.

This completes our presentation of the TPayloadStream class. In the final part of the article you can download some demo code and provide feedback to the author.