Copyright (c) 1997 by the University of Washington. All rights reserved. This is a proprietary and confidential document. Under no circumstances may this document be used, modified, copied, distributed, or sold without the express written permission of the copyright holder.

Integrating Tools Into Visual Etch

Contents

1. Introduction
2. Details of Tool Integration
3. Visual Etch Services for Tool Scripts
Appendix A. Visual Etch Interfaces

1. Introduction

This document describes the interface between the Visual Etch user interface and Etch instrumentation tools. In particular, it shows how to integrate a new tool into the Visual Etch environment, and what procedural and environmental support facilities Visual Etch provides for tools. This document is intended only for tool writers wishing to add their tool to the Visual Etch user interface. It does not describe the end-user interface to Visual Etch; that is described in detail the Visual Etch User's Guide.

1.1 Overview of Visual Etch Components and Interfaces

Visual Etch presents the user with a window- and menu-based environment for controlling instrumentation and optimization experiments on executables. It allows the user to specify the executable and DLLs of interest, together with the script or command line parameters that should be used to run the executable. The user also specifies the tool that they wish to apply to the executable.

There are actually three components that participate to execute an Etch experiment through Visual Etch. These components commmunicate through various interfaces described in this document. The communication is through files, global variables, or procedure calls, depending on the components communicating. The three major components responsible for running an experiment through Visual Etch are:

  1. The Visual Etch user interface - This code is responsible only for the visual interaction with the user; that is, windows, menus, user input, and so on. Through the user interface, the user selects the application to process and the tool (the "action") to apply to that application. Once the user specifies the tool, the user interface prompts for tool-specific parameters. The Visual Etch user interface then produces files (project.txt, etchparams.txt, and [appname].options, described later in the document) that contain information specified by the user; these files are examined by the second component, the script coordinator.pl. The Visual Etch user interface also sets up pipes and event objects for communicating with scripts.

  2. The perl script coordinator.pl - Once the user selects an application and tool, the coordinator.pl perl script is responsible for managing the process of etching the application. In particular, coordinator.pl: (a) examines the files produced by the user interface, (b) converts the user-specified options into perl global variables, (c) calls the specified tool, and (d) after the tool finishes, copies the project files to the working directory and creates a log file to aid in debugging the etched application in case it is needed.

  3. The user-specified tool - The tool may be either a subroutine in a file included by coordinator.pl or an executable on the current path. Once the tool is invoked, it is responsible for interpreting the options that the user specified to produce and/or run an etched application. The tool needs to invoke Etch and dllwalk correctly on the application modules, respecting the options that the user specified. If the tool requires debugging information, it must run cvdump and convert the output of cvdump into a form that Etch can use. It also needs to manage the Etch cache (if it chooses to cache the results of previous Etch invocations) and report its progress to the user, including errors and warnings that Etch produces. To help tool writers manage this entire process, Visual Etch provides perl routines that take care of much of this complexity.

This document describes the steps for integrating tools into Visual Etch and the associated tool scripts used by Visual Etch. To integrate a tool into the user interface, the tool implementor must understand five main interfaces:

  1. An interface for the tool writer to add a tool to the Visual Etch user interface; adding a tool to Visual Etch allows Visual Etch to display that tool's name as an option, so that the user can select the tool and Visual Etch can invoke that tool.

  2. An interface for the tool-writer to define tool-specific options, so that Visual Etch can prompt the user for those options when the user selects the tool.

  3. An interface for the Visual Etch user interface to pass the user-specified, tool-specific options to coordinator.pl and/or other tool scripts that manage the tool execution process.

  4. A set of library routines that Visual Etch provides to manage project, debug, and Etch files, and to provide feedback from the tool scripts to the user interface.

  5. An interface between the tool and the user scripts that manage the application execution process.

1.2 An Example of Adding a New Tool to Visual Etch

In this section, we briefly walk through the steps of integrating a new tool into the Visual Etch user interface. For this example, we integrate an unaligned access counting tool, as described in the
Etch Tool Writer's guide. In the sections following this example, we describe in more detail the various interfaces and the Visual Etch environment in which tools and scripts execute. The steps for adding a new tool to Visual Etch are as follows:
  1. Write and compile the tool instrumentation and runtime routines

    For the unaligned access counting tool, create unaligned-inst.cxx and unaligned-rt.c using Visual C++. Compile these two files into unaligned-inst.dll and unaligned-rt.dll.

  2. Copy the tool instrumentation and runtime DLLs into the etch\apps\bin directory.

    Copy unaligned-inst.dll and unaligned-rt.dll into etch\apps\bin.

  3. Write a script that runs the Etch experiment with the tool and specified application, and provides feedback to the user; Visual Etch provides services to simplify the writing of the script, as described in Section 3.

    We're going to call this tool "unaligned", so create a perl file called unaligned.pl in the etch\bin\scripts subdirectory, with a perl function called unaligned. Following is an example of the unaligned perl function. While the details of the calls are described later in this document, the comments in the function indicate the basic steps that it is performing. This code can be used as a template for other tools. Notice that among other things, this perl script invokes Etch, manually adjusts the progress bar displayed in the Visual Etch "Running Experiment" window at various points, and invokes a function to visually display the tool output.

      
         sub unaligned {
             local($debugfile) = "";
    
    	 # Create checklist to tell user what's happening
    	 #   Etch $EXENAME and associated DLLs -> we're etching the application
    	 #   Run $EXENAME -> Run the etched application
             &mCheckList("Etch $EXENAME and associated DLL's,Run $EXENAME");
        
             # Determine if the user wanted to etch the application 
             # ("Execute Action" or "Etch Only")
             if (ACTION_SHOULD_ETCH()) {
    	     # Yes.  The user wanted to etch the application
    	     # Clear the working directory of etched application and state
    	     # from previous runs
    	     SetupWorkingDir();
    	
         	     # Etch the application, passing application name, tool name, 
                 # and list of DLLs to be etched	
    	     &etch($EXENAME,"unaligned","","","",@ModuleNames);	
             }
    	 
             # Check off "Etch EXENAME and associated DLL's"
             mCheck(0);
        
             # Increase the progress bar.  We're 50% done
              mProgress(50);
    
             # Find the name of the processed application.  It's either
             #   app-etch.exe or app-patch.exe. 
             $outname = FindProcessedExecutable($EXENAME);
    
             # Determine if the user wanted to run the application 
             # ("Execute Action" or "Run Only")
             if (ACTION_SHOULD_RUN()) {
    
        	     # Run the etched binary.
     	     &run($outname);
    
    	     # Show the result in a table format
    	     mTable("Unaligned Accesses","$WORKINGDIR\\unaligned.output");
             }
    	   
           # Check off "Run EXENAME"
           mCheck(1);
        
           # We're 100% finished
           mProgress(100);
        
         }
         
         # need to put a 1 at the end of a script file because the perl 
         # require mechanism requires this. 
         1;
         

    For details on other perl functions that the tool scripts can call, see Section 3.

  4. Copy the perl script to the directory etch\apps\bin\scripts

    Visual Etch automatically looks scripts to run a tool in the directory etch\apps\bin\scripts. All files that end with the suffix .pl in the directory are searched for one that contains the subroutine called with the same name as the tool, in this case sub unaligned.

  5. Create a < tool>.desc file in etch\apps\bin\desc.

    When Visual Etch starts up, it reads all the files with the suffix .desc in the etch\apps\desc subdirectory. These files describe what should be contained in the "Action" list in the Visual Etch window. In this case you will create a file called unaligned.desc:

          #
          # This is the description file for the unaligned tool
          #   All the fields defined in this file must be present for
          #    all tools in desc files.  
          #
          # One desc file may contain descriptions for multiple tools
          #   as long as they are delimited by the index field
    
          #
          # Index delimits different tools.  This indicates that
          #   a new tool is will be described 
          #
          # Index also tells where you want the tool to appear in the tool list.  
          # Since the tool is still experimental, we put in 3001 which causes it
          #   to be clumped with "Cutting Edge Tools" in the tool list.
          Index = 3001
    
          # Metric - the string you wish Visual Etch to display to the user in
          #   the tool list
          Metric = Unaligned Accesses
    
          # ToolName - name of the tool
          ToolName = unaligned
    
          # User options - you can specify a table of tool-specific options that
          #   Visual Etch will request from the user.  
          # "null" in this case because the unaligned tool has no special options
          UserOptions = null
    
          # Help - A help text that Visual Etch will display in the status bar
          #   when the tool is highlighted.
          Help = Count unaligned accesses
          

  6. Write a < tool>-toolinfo.txt file in the etch\apps\bin directory.

    < tool>-toolinfo.txt is a text file describing for the user the output of the tool and, if necessary, how the output should be interpreted. In this case, Visual Etch looks for unaligned-toolinfo.txt to display together with the tool output.

  7. Restart Visual Etch

    Visual Etch needs to be restarted for the changes to the tool list to be reflected. Quit all instances of Visual Etch and then start Visual Etch again. Visual Etch should now show your tool in the tools list. If it does not, check to ensure that no other instances of Visual Etch are still running.

2. Details of Tool Integration

This section describes, in more detail, how to integrate a tool and tool script into the Visual Etch user interface, including the files and variables used for communication between Visual Etch and the perl scripts.

2.1 Adding a tool to the Visual Etch "Action" list

The tools known to Visual Etch and displayed in the "Action" list in the main Visual Etch window are described in .desc files in the etch\apps\bin\desc subdirectory. The fields of the .desc files define different fields and are of the form:
      < field> = < value>

The required fields are:

Index
Determines the order in which the tools appear in the "Action" list box under Visual Etch.
Metric
The name to be displayed in the list box for selection.
ToolName
The name of the tool that Visual Etch will call if the associated Metric is selected. The ToolName should specify a subroutine in the the scripts in the etch\apps\bin\scripts directory.
UserOptions
The name of a table in etch\apps\bin\params that is filled with options for the user to pass to the tool. See Telling Visual Etch to ask users for tool specific options.
Help
A longer string that describes the tool; the Help string is displayed by Visual Etch to give the user more information about the tool.
For an example, click here.

2.2 Adding perl scripts for Visual Etch to use

coordinator.pl loads in all the files with the suffix .pl in the etch\apps\bin\scripts subdirectory. The files in the directory are required by coordinator.pl, so any functions defined by these files are automatically incorporated into coordinator.pl through the perl require mechanism.



2.3 Telling Visual Etch to ask users for tool-specific options


When adding tools to Visual Etch, you can set up options for the user to specify by creating a .params file in the etch\apps\bin\params subdirectory. Visual Etch will prompt the user to specify these options before running the tool script. These options are read into the %PROJECT associative array and accessed through:
     $value = $PROJECT{[tool],[key]}

Each option in the .param file is required to have at least six fields of the form:
      < field> = < value>
The fields have the following data:

Index (required)
The order of the different options for which to prompt the user. This field signals the beginning of a new option definition.
Description (required)
The description of the option; this is what the user sees in the first column of the tool option grid.
Value (required)
The default value of the option that the user sets.
Key (required)
The key that is used to set the particular option - i.e., it identifies which value is being supplied. The key is used to identify this particular option through %PROJECT to the tool script. For example, if the cache tool has the toolname cache, and the key for the data cache size is d, then this value can be accessed through $PROJECT{"cache","d"}.
HelpText (required)
The text that Visual Etch prints on the status bar when the user clicks on the particular parameter.
Options (required)
There are three options for this field:
  1. Blank or the string null - indicates that the option can be freely entered by the user through a text box.
  2. the string FIXED - indicates that the option cannot be set by the user and is there only for information purposes.
  3. a comma delimited list - specifies the list of options that the user can set and will appear as a combo box.

Additional fields for default configurations(optional)
The other fields contain default values for different default configurations the user might be interested in. A drop-down menu is provided to help the user select one of these default parameters. For example, an additional field for machine parameters may be called "Pentium 100" and each option would contain the field "Pentium 100" with values appropriate for the Pentium 100 processor.

Note: All options must have the same additional fields and that each of the fields cannot be blank.

For an example of how this works, look in the file CPU.params in the directory etch\apps\bin\params.

2.4 Information Visual Etch passes to the tools


Visual Etch passes along particular information that the user has specified to an external user-specified routine. The user specifies the routine and its inputs though dialog screens. Before Visual Etch calls the routine, it sets up several environment variables as well as several project files: project.txt, etchparams.txt, and for each module a [application].options file. Details of these files can be found in
Appendix A

Although the tool scripts can parse the information in these files and the environment variables, coordinator.pl parses these files and environment variables and puts them into perl global variables as a convenience for tool scripts.

2.4.1 Global Variables defined for perl scripts

The following global variables are defined in coordinator.pl and can be used by the perl tool scripts. They are derived from the contents of the file project.txt and from environment variables set by Visual Etch.

Directories

The following variables contain the directories where the script can find the application and Etch components.

  1. $PERLDIR - the directory where the Perl executable and DLLs live
  2. $ETCHROOT - the root of the Etch application tree
  3. $ETCHBIN - the directory where the Etch application lives
  4. $ETCHTOOLS - the root directory for $ETCHTOOLS\bin, which contains the instrumentation and runtime DLLs for the tools
  5. $PROJECTDIR - the root project directory
  6. $WORKINGDIR - the directory in which the tool can write its results and temporary files. Etch writes the etched and patched modules into this directory.
  7. $TEMPDIR - the temp directory (same as %TEMPDIR%)
  8. $CACHEDIR - the project cache directory

Application Information

The following variables contain information about the application that the user wishes to etch.

  1. $EXENAME - the full path name of the executable that the user wants to etch.
  2. $SCRIPTNAME - the script that the user wants to run on the executable.
  3. $COMMANDLINE - the command line argument that the user passed in. If $SCRIPTNAME is not empty, then this is the command line for the script. Otherwise, it is the command line for the application.
  4. @ModuleNames - an array containing the names of the modules that the application uses

Project Information

The following variables contain information about the project and the options that the user chose with the project.

  1. $TOOL - the name of the tool that the user wants to run
  2. %PROJECT - an associative array that contains all the information in project.txt. The information is accessed through $PROJECT{[tool],[key]}.
  3. %ETCHPARAMS - an associative array that contains all the information in etchparams.txt, which gives the parameters that the user specified for each module. The information is accessed through $ETCHPARAMS{[modulename],[optionname]}.
  4. $PROC_OPTIONS_FILE - full path to the file that contains user specified selective etching options.

Miscellaneous

  1. $VISUAL_ETCH - this is set to 1 if coordinator.pl is running under Visual Etch, undefined otherwise
  2. $ETCHERRORLOG - the name of the file where Etch writes its error messages
  3. $ETCHWARNINGLOG - the name of the file where Etch writes its warning messages
  4. $ETCH_UNIQUE_OUTPUT - indicates whether or not the output of the tool should be unique per process (useful for etching applications composed of multiple processes)
  5. $TRUE - 1
  6. $FALSE - 0

Machine parameters

The following variables define machine parameters for tools such as cache simulators. The script coordinator.pl reads each parameter from the file project.txt and creates an identical global variable, thereby facilitating access to the parameters by tool scripts.

  1. $SPEED_MHZ - speed of the simulated machine in Mhz
  2. $IPERCYCLE - average instructions per cycle
  3. $L1ISIZE - size of the L1 instruction cache
  4. $L1DSIZE - size of the L1 data cache
  5. $L2SIZE - size of the L2 cache
  6. $L1IASSOC - associativity of the L1 instruction cache
  7. $L1DASSOC - associativity of the L1 data cache
  8. $L2ASSOC - associativity of the L2 cache
  9. $TLBSIZE - size of the TLB
  10. $TLBPENALTY - penalty of a TLB miss
  11. $USEWB - whether or not the L2 cache uses a write buffer to buffer writes
  12. $L1DWPOLICY - whether the L1 data cache uses a WriteThrough or WriteBack policy
  13. $L2WPOLICY - whether the L2 cache uses a WriteThrough or WriteBack policy


3. Visual Etch Services for Tool Scripts

This section describes the services that the tool scripts can use to run Etch, run the executable, and show the results of the experiment.

3.1 Support subroutines for tool implementation

This section describes the predefined perl subroutines to help the tool writer implement their tool. Although there are many routines that the tool scripts can use, most tools will only need the following routines:
ACTION_SHOULD_ETCH()
to determine if the script should run etch
ACTION_SHOULD_RUN()
to determine if the script should run the etched application
SetupWorkingDir()
to clear the state of the working directory from the results of previous runs.
cvdump(outname,lownum,highnum)
to extract debug information from the program modules
etch(exename, toolname, debugfile, extraflag, stamp, @ModuleFNames)
to call etch with the proper parameters for the program modules
run(baseExecutable)
to run the etched binary.
Visual Etch callback routines
to reflect tool progress and show output to the user
An example tool script is here

3.1.1 Subroutines to run Etch

These are routines that encapsulate the complexity of etching a full application. Most tools will just need the etch subroutine described below; some tools, however, may need behavior different than that caused by the etch subroutine, and these tools will need to call the individual subroutines that the etch subroutine calls in a different way. These tools are defined in scripts\etch.pl file.

etch(exename, toolname, debugfile, extraflag, stamp, @ModuleFNames)
This is the main routine that most tools will call. It takes the application defined by exename and the associated modules given by the list @ModuleFNames and etches each one depending on the contents of the global variable %EtchParams. The debugfile parameter specifies the name of a file containing debugging information for all of the modules for this application. The extraflag parameter specifies any extra flags that the tool needs to pass to Etch. The stamp parameter specifies a string uniquely identifying a version of the etched modules; this is used for verifying that cached etched modules are up to date. The etch subroutine takes care of patching the import tables of etched and patched programs. It takes care of caching from previous runs of Etch to save time. The routine returns the name of the newly etched executable.

EtchEverything(debug_directory, exename, toolname, appname, debugfile, extraflag, stamp, @ModuleFNames)
This routine is called by the etch subroutine described above. It etches the executable and modules named in ModuleFNames and puts them in the current directory. (Typically, the current directory is the EtchedModules folder in the experiment directory.) The parameter debug_directory is the directory in which to find the debugfile. The routine gets the Etch parameters from the associative array %EtchParams. It does not modify (i.e., patch) the modules for which SHOULDETCH is set to "no" in %EtchParams.

etchone(modulename,toolname, exename, etchedname, debugfile,extraflag, stamp)
This routine is called by EtchEverything, once for each module; it etches a single module with arguments built from the argument list passed in and those defined in %ETCHPARAMS. If an up-to-date version of the etched modules exist then it is not etched, thus assuming that the initial etched file is still valid.

CopyEtchedFilesFromDirectory(subdirectory, @ModuleFNames)
Goes through the specified subdirectory and copies into the current directory the etched modules named in @ModuleFNames that have the parameter SHOULDETCH set to "yes" in %EtchParams. (Typically used to copy etched files from the EtchedModules folder to the experiment directory.)

ComputeEtchFlags(modulename, toolname, appnams, extraflag, stamp, outname)
Returns the command line flags for a call to Etch for modulename with the parameters passed in to ComputeEtchFlags and with those defined in %ETCHPARAMS.

PatchUnEtchedFiles(@ModuleFNames)
Goes through the files given in @ModuleFNames that have not been etched and patches the calls to kernel32.dll with calls to etchwrap.dll, in order to capture loads to unetched versions of etched modules.

PatchImportExportTables(dllmaptable_name, @ModuleFNames)
Goes through the files given in @ModuleFNames and changes the import and export tables to refer to etched and patched files instead of the original unetched and unpatched files. dllmaptable_name is the name of a file where the routine can store its intermediate file, which maps the original names into the etched and patched names.

GenerateDLLWalkTable(outfilename,@modulenames)
Generates a map from the original names listed in @modulenames to the etched or patched names, depending on whether the etched or patched files are in the current directory. It puts this table in the file outfilename.

dllwrap(originalname,newname)
Modifies the import table of the module named originalname to change the routines calling kernel32.dll to load modules to instead call etchwrap.dll to correctly load etched and patched modules.

dllwalk(dllwalkarg,modulename)
Runs dllwalk on modulename with the arguments given by dllwalkarg.

CheckEtchStamp(modulename,etchedfile,commandline)
This routine compares the Etch command line that is stored in the etchedfile with the command given by the commandline parameter. The comparison is done to verify that the Etch parameters specified in commandline are the same as those used for the invocation used to create the cached copy of the module. Also checks if the Etch version numbers of both files would be the same. Deletes etchedfile and returns FALSE if either do not match. Returns TRUE otherwise.

GetWarningLog
Reads the warning log file defined by the global variable $ETCHWARNINGLOG and returns with an array of the contents of the file.

GetErrorLog
Reads the error log file defined by the global variable $ETCHERRORLOG and returns with a string containing the error.

3.1.2 Subroutines to run applications or scripts

These routines help with running the application or scripts. Most tools will only need the run routine, but other tools may require something different and would have to call the other routines in a way different from run.

run(baseExecutable)
Runs baseExecutable with the script specified in $SCRIPTNAME (if $SCRIPTNAME is specified) and creates two batch files for running the baseExecutable from the commandline (etch-setenv.bat and etch-run-me.bat) in $WORKINGDIR.

runOriginal(exename)
Runs the original exename instead of the etched one. If $SCRIPTNAME is set, then the script $SCRIPTNAME is run on exename.

CreateTargetDirBatchFile(targetdirectory,base_executable, commandline_arguments)
Creates two files in targetdirectory for running the etched application: etch-setenv.bat and etch-run-me.bat. The routine reads the entire environment block and dumps instructions on how to set the environment into etch-setenv.bat. etch-run-me.bat calls etch-setenv.bat to set the proper environment for running the application, then executes base_executable commandline_arguments .

setupRun(baseExecutable)
Sets up the environment and path to run the executable or script properly, and calls CreateTargetDirBatchFile($WORKINGDIR,baseExecutable,$COMMANDLINE). Returns the original value of %PATH%.

genCommand(exename)
Returns the proper command to execute under the current environment. If SCRIPTNAME is set, returns a command to run SCRIPTNAME, otherwise returns a command to run exename

genScriptName(exename,script,commandline_arguments)
Returns the command to execute the script on exename with the script taking commandline_arguments

finishRun(oldPath, errorcode, exename)
Cleans up after the execution of exename. Restores the path to oldPath and checks the errorcode to see if there was an error in running exename.

3.1.3 Auxiliary routines

Following are additional routines that Visual Etch defines to help with etching.

cvdump(outname,lownum,highnum)
Finds the debug information for the current executables and the DLLs and puts it in outname. lownum and highnum give the percentage of the total work that the debug phase takes up. For example, if before the debug phase, 20% of the work is done, and after the debug phase 50% of the work would be done, lownum would be 20, and highnum would be 50. Returns $TRUE if there is debug information, $FALSE otherwise.

WriteLogFile(logdir)
creates etch.log in the directory named by logdir. This is created automatically by the main routine in coordinator.pl. This is defined in the file scripts\log.pl.

SetupWorkingDir()
sets up the working directory to ensure that it's ready to be run by etch. This entails moving etched files into the subdirectory EtchedModules and deleting everything else in the working directory. It then creates the file etch.config which contains the original location of the etched executable.

FindProcessedExecutable(executable)
given the name of an executable, looks in the working directory for an etched or patched version of it and returns the name of the etched or patched version.

ACTION_SHOULD_ETCH()
returns 1 if the user specified an action that requires that the tool should produce a new etched executable.

ACTION_SHOULD_RUN()
returns 1 if the user specified an action that requires that the tool should run the etched application.

CheckAction(ACTION)
checks action to see if it's a valid action: ETCH_ONLY, RUN_ONLY, or EXECUTE_ACTION.

3.1.4 Callbacks to Visual Etch

Following are routines that are used for sending messages to Visual Etch. These routines are defined in message.pl.

SendMessage([messagestring])
sends the message string given by [messagestring] to Visual Etch. Usually, however, the tool writer will want to call the Perl functions that encapsulates the message sending rather than SendMessage (e.g. call mShowMessage(message) instead of SendMessage("ShowMessage message"))
mShowMessage($message)
Shows a message in the status bar and in the Run Details dialog box.
mShowMessageArray(@message_array)
Shows multiple messages in the Run Details dialog box.
Table($filename)
Displays a table. The file should be a comma-delimited file with each row having the same number of columns.
mQuickGraph($Title,$GraphType,$filename)
Displays a graph. The file to be graphed should be a table in a comma-delimited file. The first row should contain column headings and the first column should contain labels for the different values. If there are only two columns, the second column serves as the data values. If there are more than two columns, the second column serves as X-values and the other columns serves as series of different Y-values (e.g., for bar graphs). GraphType is an integer between 1 to 11 inclusive, indicating the style of presentation, as defined below (GraphType defaults to 3 if it is passed a number not between 1 and 11, inclusive):
  1. 2D pie
  2. 3D pie
  3. 2D bar
  4. 3D bar
  5. Gantt
  6. Line
  7. Log/Lin
  8. Area
  9. Scatter
  10. Polar
  11. High/Low chart (i.e. stock type charts)
mDisplayFile($title,$file)
Shows the contents of the file in a text box.
mProjectInfo($tool,$key,$value)
Saves the Tool-Key-Value triple as part of the project information. The tool can use this to keep state between runs.
mProgress($percent_progress)
Displays the percent of work completed, as specified.
mCheckList(@checklist)
Provides Visual Etch with the checklist shown in the Run dialog.
mCheck($item)
Indicates to Visual Etch that this item is done.
mPause($message)
Shows a message box with "message" and waits for the user to press enter to continue.
mError($message)
Indicates that Visual Etch should pop-up a message telling the user that the error condition specified in $message has happened.

3.1.5 Utility functions

These functions help the tool writer with common tasks. The routines are defined in the file scripts\util.pl

EtchedSystem(command)
Like the Perl system command, except that it takes special care in Win95 to insure that paths with spaces are handled correctly. Note that these paths still need to be quoted. EtchedSystem just insures that the system call returns synchronously.

CachedSystem(command,outputfile,@dependencies)
Checks to see if outputfile is newer than the files named in dependencies. Does nothing if outputfile is newer, otherwise, deletes outputfile and runs command.

CheckFile(outputname,@inputnames)
Returns true if outputname is newer than all of inputnames, otherwise returns false

Trim(string)
Strips the leading and trailing spaces from the string

UPCASE(string)
Returns the string with all uppercase letters

locase(string)
Returns the string into all lowercase letters

EnQuote(string)
Returns the string surrounded by double quotes. Useful for specifying the commandline for file names with spaces embedded in them.

Strip2BackSlashes(string)
Returns the string with all double backslashes converted into single backslashes

EtchedName(filename)
Returns the name of an etched version of the file (e.g., for "file.ext" it would be "file-etch.ext")

PatchedName(filename)
Returns the name of a patched version of the file (e.g., for "file.ext" it would be "file-patch.ext")

BaseName(filename)
Takes a full path including the file and returns the name of the file without the directory

BaseDir(filename)
Takes a full path including the file and returns the name of the directory without the file

StripName(filename)
Strips the extension from filename (e.g., for "file.ext" it would return "file")

Extension(filename)
Returns the extension of the file

DebugPrint(string)
Prints the string to stderr with the prefix "DEBUG:"

PrintTime(FILEPOINTER)
Prints the time to the file passed in as FILEPOINTER

IsNT()
returns true if the system is NT

CombineTextFiles(dest,@sourefiles)
Reads all the files listed in @sourcefiles, and concatenates them into dest

which(filename)
Returns the full pathname of the file called filename which would be loaded by a GetModuleHandle call. This searches the current directory for the file, then searches the directories defined in the path environment variable.

CopyFile(source,dest)
Copies the file from source to dest taking care to handle long filenames with spaces correctly

MoveFile(source,dest)
Moves the file from from source to dest taking care to handle long filenames with spaces correctly and checking to make sure that the move operation succeeded

ReallyUnlink(filename)
Tries to delete filename.

UnlinkRegexp(regular_expression)
Deletes files that match regular_expression. Implicitly adds ^ and $ around regular_expression to only match full strings. Dies if a delete operation fails.



3.2 Caching Previously Etched Modules

The
routines to help run Etch provided by Visual Etch create a cache of previously etched modules under the EtchedModules subdirectory to speed up subsequent runs of etch. To keep this cache coherent three tests are performed:
  1. The cached etched file must be newer than the original file, etch.exe and the tool instrumentation DLL.
  2. The cached etched file must have been generated by the same version of Etch as that currently running.
  3. The arguments used to invoke Etch on the cached etched file must be identical to the arguments that would be invoked to etch the current file.

IMPORTANT: Sometimes tools may have metadata that cannot be captured in these three categories, but nevertheless affects how a file is etched (e.g. a user parameter to determine which Procedure to instrument). Under these circumstances, it is the tool writer's responsibility to encode this information with a stamp that can become a part of the Etch arguments (e.g., through the -stamp option).

For an example of how this works, look in cache.pl in the %ETCHBIN%\scripts subdirectory.

3.3 Miscellaneous

Communication between scripts to run applications and Visual Etch scripts

Scripts to run the application should use the environment variable NEWEXENAME to find the executable name of the executable to be run. This environment variable is initialized by the run subroutine before it executes the script.

Making tools ready for Visual Etch

The Visual Etch scripts usually expect that the output of the programs are placed in %WORKINGDIR%. A utility for getting this name is in the apps\libutil subdirectory etchOutputName. For an example look in ProgramAfter in unaligned-rt.c. The tool runtime DLL should also respect the ETCH_UNIQUE_OUTPUT environment variable which tell it that each process should output to a unique file.



Appendix A. Visual Etch Interfaces

Most tools will be written in Perl and hence would only need to understand the Perl global variables and routines that provided by Visual Etch. However, tools written in other languages have to understand the interfaces of
how Visual Etch passes information to the tools and how Visual Etch shows user feedback.

A.1 Information Visual Etch passes to the tools


Visual Etch passes along particular information that the user has specified to an external user-specified routine. The user specifies the routine and its inputs though dialog screens. Before Visual Etch calls the routine, it sets up several environment variables as well as several project files: project.txt, etchparams.txt, and for each module a [application].options file.

A.1.1 Visual Etch environment variables

VISUAL_ETCH
set to 1 to indicate that Visual Etch is running.
EXENAME
the full path name of the executable that the user specified
DLLNAMES
A comma delimited list of the names of the DLLs that the user specified.
PROJECTDIR
The project directory
WORKINGDIR
The experiment folder, which is a subdirectory of the cache directory. All output files go into this directory. This is also the directory in which the tool will find itself when it is called.
TEMPDIR
A temporary directory that the tool can use to dump temporary files; typically a subdirectory of the system temp directory.
TOOL
The tool that the user specified.
COMMANDLINE
The command line argument that the user specified for the executable.
SCRIPTNAME
The script that the user specified to run the executable.

A.1.2 Visual Etch project files

Before Visual Etch runs coordinator.pl and the tool script, it creates several project files that contain the options that the user specified through the UI. For boolean options, the option values are encoded as the strings "Yes" and "No", e.g.: "winword.exe, SHOULDETCH, Yes".

  1. project.txt
  2. Visual Etch creates a project file called project.txt in the project directory, which contains parameters for the whole project. The file contains three columns (comma delimited), where each row contains the triple [tool],[key],[value]. [tool] tells which tool created the key, followed by a [key],[value] pair. The [key],[value] pair is created either through user options or through a ProjectInfo message, sent from the tool back to Visual Etch.

    The rows with tool name Project are values set by Visual Etch. These keys are:

    ACTION
    action the user specified: EXECUTE_ACTION, ETCH_ONLY, or RUN_ONLY
    TOOL
    the tool that the user chose
    EXENAME
    the name of the application main executable
    MODULENAMES
    a comma delimited list of all the modules that the application uses
    ROOTMODULES
    a comma delimited list of explicitly loaded modules
    SCRIPTNAME
    name of a script to run the application
    COMMANDLINE
    The command line that the user specified for running the script or executable. The command line is for the script if SCRIPTNAME is not blank, otherwise it's the command line for the executable.
    SELECTIVEETCHING
    whether or not to use selective etching
    DLLWRAP
    whether to patch calls to LoadLibrary and LoadModule to the special version in provided in etchwrap
    ETCH_UNIQUE_OUTPUT
    whether tools should create a unique output file for each etched process. This is useful for applications that use multiple executables, so that multiple processes do not attempt to write the same output file.
    DATE
    the date the action is invoked
    TIME
    time when the action is invoked
    ETCHVERSION
    the version of Etch when Visual Etch was started
    VETCHVERSION
    the version of Visual Etch
    PROJECTDIR
    the project directory
    CACHEDIR
    location of the cache directory. Temporary non-tool or non-executable specific files usually show up in this directory.
    WORKINGDIR
    the experiment folder. Tool and application specific files will end up in this directory.
    TEMPDIR
    the temp directory (the same as %TEMPDIR%)

  3. Machine Parameters
  4. Machine parameters can be accessed through the MachParams records in the project.txt file. (These are not necessarily the parameters of the machine on which Etch is executing; they provide a mechanism for the user to specify machine parameters to simulation tools doing on-the-fly simulation using Etch. The perl scripts managing the tool execution define global variables from these parameters to make them easier for the simulation tools to use: see the Section on Global variables defined for perl scripts.) Each row contains a [MachParams,key,value] triple, where the keys are:

  5. etchparams.txt
  6. etchparams.txt contain the per module etching options that the user specified through the Modules dialog box. This file contains triples of the form [Module name, Etch parameter, value]. The Etch parameters are:

    SHOULDETCH
    Whether or not to etch the module
    SHOULDPATCH
    Whether or not to patch the module
    CODEDISCOVER
    Whether or not to use aggressive code discovery
    FASTCONTEXT
    Whether or not to use fast context switch
    SAVEFP
    Whether or not to save the floating point state when the etched module call a tool procedure
    ORIGBASE
    Whether or not to load the module into the original base address
    ORIGNAME
    Whether or not to use the original name of the module. If not set, -etch or -patch is appended to the module name for etched and patched modules.

  7. [application].options
  8. For selective etching, Visual Etch creates an [Application].options file that lists the procedures that are contained in the application and whether or not these procedures should be etched. The file contains triples of the form [option,module,procedure]. Module and procedure gives the module name and procedure name respectively. Option can be either "+" or "-"; "+" means etch the procedure, "-" means do not etch the procedure.



A.2 Asking Visual Etch to show user feedback

Visual Etch communicates with the tool script through an anonymous pipe and events. Visual Etch passes the handle for the write end of the anonymous pipe in the environment variable PipeHandle, and it passes the event names through the environment variables serverWait and clientWait. To send a message to Visual Etch, simply write a message to PipeHandle, signal the event serverWait, and wait for the event clientWait. Tool scripts that are not written in perl need to send these messages explicitly; however, tools written in perl can use the perl subroutines defined in message.pl.
Following is the list of messages that Visual Etch recognizes and the action performed by Visual Etch on each message. A description of these messages is given in Section 3.1.4.

ShowMessage [message]
Table [file to show as a table]
QuickGraph [Title],[GraphType],[file to be graphed]
DisplayFile [title],[file]
ProjectInfo [tool],[key],[value]
Progress [% of work done]
CheckList [comma delimited list of items to be checked off]
Check [item number]
Pause [message]
Error [message]
Done [0=fail,1=success],[message]
Indicates to Visual Etch that the script has finished doing its work. This is called automatically by coordinator.pl, so the scripts don't need to call it directly.