Question: What are the system requirements for running Etch?
Answer: To run Etch, you'll need Windows NT 4.0 or Windows 95, 32 MB of physical
memory, and a large swap file. We estimate that a large binary like
WinWord needs at least 150 MB of swap space.
Question: What are the system requirements for writing tools in Etch?
Answer: In addition to the requirements listed above, you'll need:
Microsoft Visual C++ to develop the tools and
Microsoft Access to modify the database tables that the user
interface uses. You will also install Visual Basic to guarantee that you have all
the files needed for Visual Etch.
Answer: You can see the details of the current Etch execution
in the "Check List" and "Run Details" windows of the Visual Etch "Running
Experiment" window. In addition, you can see in detail exactly
what command Etch is executing by selecting "Status Windows | Console.."
from the menu options in the Running Experiment window. Be patient, sometimes
it takes a while for Etch output to be reflected in the
console window. You can also look in the task
manager window to see if Etch is making progress.
Answer: Yes. However, you'll need to make sure that you are
not using the same project, nor sharing temporary directories. By
default, Visual Etch places its project directory in a temporary
directory under the system directory that is called username.temp (for
a specific "username"), so this normally is not an issue.
Question: What kinds of binaries can I etch?
Answer: You can etch
32-bit PE format Windows binaries, including both executables and
DLLs. Certain kinds of instruction sequences found in hand-written
assembly language will fail when Etched, but these sequences occur
rarely in practice. Examples include self-modifying code, code that
jumps into the middle of an instruction, and code that uses pointers
to instructions to refer to data.
Question:Does Etch work on 16 bit codes?
Answer: No.
If you notice that a program is using a DLL called something like
"foo16.dll" then you have a strong clue that it is relying on 16-bit
code; since Etch does not support 16-bit code, that DLL should be
omitted from the list of DLLs to be etched and patched (using the Modules window).
Question: Can I etch an already-etched binary?
Answer: Yes, although this is not currently supported directly through the Visual
Etch interface. Etching an etched binary
requires special naming and special modifications to the tools you
will use.
Question: Can Etch be used with multithreaded programs?
Answer: There should be no problem using Etch on multithreaded
programs, provided that the tool to be used is thread safe.
Specifically, if your tool manipulates shared data structures,
then the tool should enforce whatever locking is required to access
those data structures.
Question: Can I run Etch on a multiprocessor?
Answer: Probably, but we have not tried it yet.
Question: What additional
link flags do I need to specify when using MSVC 5.0
so that I can produce object files that Etch can process?
Answer: If you are linking from the
command line, then you need to add the
flag "/fixed:no" to your link command. If you are linking from within
msdev, then you need to add the flag to your project settings: invoke
the "Project->Settings" menu command, select the "Link" tab in the
dialog that appears, and then add the flag "/fixed:no" to the "Project
Options" text window at the bottom right of the window.
Question: How do I instrument a program that relies on multiple
executables (.exe files)?
Answer: The basic strategy is to fool Visual Etch into thinking
that the union of all the executables and DLLs in which you are
interested is a single executable. Here's how to do it.
Suppose we have two programs, helper.exe and master.exe, where
running master.exe causes the execution of helper.exe as a helper
process. Often, the two programs communicate through common DLLs. We
must determine the set of DLLs used by both helper.exe and master.exe
(call these sets helper-dlls and master-dlls, respectively). We then
extend master-dlls to include all of the DLLs in helper-dlls, as well
as helper.exe itself. We then run our experiment on master.exe, which
will cause all of helper.exe's components (both DLLs and helper.exe)
to be included in the experiment. We have to be a bit careful about
how we invoke master.exe to ensure that the results from helper.exe
and the results from master.exe end up in separate files (since each
has its own ProgramBefore and ProgramAfter instrumentation callbacks).
The specific steps we need to follow for helper.exe and master.exe are
as follows (obviously, you replace these names with the names of your
programs):
Question: I want to instrument a DirectDraw application and the DLLs that it
loads. Do I have to do anything special?
Answer: DirectDraw applications use two executables, so we will
need to rely on the above procedures above for running applications
with more than one .exe file. Once you've done the first
DirectDraw application, subsequent ones are quite easy. Follow these steps:
You may continue to use the file ddhelp-dlls for other DirectDraw
applications. Note that this file could become
obsolete if you install a new program that uses DirectDraw, but relies
on a later version of ddhelp.exe. If this happens, you should see
that unetched DLLs are being loaded during your program's execution and
you should follow the steps above to recreate ddhelp-dlls.
Answer: By default, Etch transforms every DLL used by the
program, say "foo.dll", into either "foo-patch.dll" or "foo-etch.dll".
The "-patch" DLL is created when the user specifies that foo.dll is
not to be etched. Although the code for the DLL is not changed, we
modify the imports used by the DLL, therefore we produce a new version
with a new name. The "-etch" DLL is created when Etch modifies the
code within the specific DLL.
Question: What if I don't want to change the
names of programs or DLLs when they have been
etched?
Answer: The UI supports this directly from the Modules screen;
select "Use Original Module Name" for the component you wish not
to rename. However, if you do this for DLLs, then you will get in
trouble if your analysis routine also uses the DLL, because your analysis
routine will end up instrumenting itself (which will be like a dog
chasing its tail). There is no problem doing this for executables.
Question: Suppose my runtime analysis code relies on a function
in a DLL that has been etched. What happens?
Answer: By default, Etch renames DLLs (eg, foo.dll becomes
foo-etch.dll). In addition, your analysis code will be linked against
the unetched version of the DLL. Consequently, your analysis code will
be using a different DLL than your program. As long as the DLL is
written in such a way that it can withstand being loaded twice (with
different names at different address) into the same address space,
then everything should work fine. If, though, the DLL is sensitive to
its name or location, or somehow shares with other DLLs through a
non-standard interface, then this multiple loading of the same DLLs
may cause problems.
Question: My program (and the DLLs it links with) loads lots of DLLs
explicitly with the Win32 LoadLibrary call. Will it run properly
after being etched?
Answer: Yes, although you will have to help Etch
identify the set of DLLs used by your program. While Etch can
automatically find the set of DLLs that are loaded statically by your
program, there is no systematic way to find every dynamically-loaded
DLL. The DLLwatch tool is, however, an effective way to do this.
Question:What happens if I fail to tell Etch about DLLs used within my program?
Answer:
Problems can be caused when a DLL references another DLL that was
neither etched nor patched (e.g., a DLL dynamically loads another DLL
that was not listed in the Modules window). This is because the
second DLL may in turn include references to other DLLs that
themselves have been etched. But, since the second DLL has not itself
been modified, it will include references to unetched versions of
those otherwise etched DLLs, resulting in two (or more) versions of
the same DLL executing in the instrumented address space.
It may be important for a program to not refer to both the
instrumented (or patched) and uninstrumented (or unpatched) versions
of the same DLL. To test for this, use the DLLwatch tool in order to
find the DLLs loaded by the application. Or, use the monitor program
to discover the set of DLLs loaded into the address space as the
program runs (you can invoke monitor using the "create command shell"
option from the UI). You should add any DLLs as appropriate to the
list of DLLs shown in the "Modules" window.
Answer: There are a variety of problems that could cause an
etched program to fail. Here, we describe some of the individual
problems you can run into and present point solutions.
> msdev target.exe
While debugging symbols are not available for the etched program, they
are available for the tool runtime code. You can simply open the
source file for the tool and set a breakpoint at the appropriate line.
In some cases it is easier to debug by
writing a standalone program that invokes
the tool directly, rather than from etch.
Here's a short list of things to check in order to get your etched
application running:
In this case, you may need to run this program from the command line.
To do this, first etch the application as normal, using the UI. Second,
open up a command window in the cache directory. Third, set your environment
variables as necessary and run "etch-run-me.bat" from that directory.
If the program executes when you do not rename the executable, then
you've solved the problem. From now on, do not rename the executable
when etching this program.
If you don't see a bad data reference, then it's possible that Etch is failing
to correctly transform one or more DLLs. You now need to determine which
DLL, when etched, is causing the problem. Again, use a simple tool
(e.g., instruction count) to make sure that the tool is not causing
the problem. To determine which DLL is bad, use whatever search
strategy you like. The simplest approach is to begin with all of the DLLs
Patched but not Etched. Then enable etching of the DLLs one at a time,
running Etch on the executable plus the
previously "known good" DLLs. When you discover that the DLL you just
added is causing trouble, disable etching on that DLL. Eventually,
you will come up with a set of DLLs that can not be included.
This failure indicates that one of two things is happening:
either you have specified a DLL that can neither be
etched nor patched, or you are specifying a DLL that can be patched,
but not etched. Both problems reflect bugs in Etch, but at different
levels. In the first case, Etch is failing to recreate the DLL
environment that NT and the program is expecting. In the second case,
Etch is failing to faithfully transform the executable. Note that the
second case could be caused by a badly-behaving executable (e.g., one
that operates on its text segment).
More than likely, though, the problem is with code discovery. By
default, Etch uses an aggressive code-discovery algorithm to find as
much code as possible in the executable and the DLLs. At times, the
algorithm may be overly aggressive, misinterpreting some data as code.
To see if this is happening, go to the Modules menu, re-enable etching
on the offending DLLs you discovered from the previous step, but
disable "Aggressive code discovery." Rerun the program. If everything
works, then you've discovered a problem with code discovery and should
contact the developers.
Answer: You don't have sufficient permission to system files and resources
required to install Visual Etch. You need to increase your
privileges
either as administrator on your NT box, or by contacting your local
system administrator.
Problem: When I start Vetch, I get an error message "Object
so-and-so not found."
Answer: You probably don't have Visual Basic installed
correctly. Reinstall VB.
Problem: The graphing option on the output does not work.
Answer: Install a current version of Visual Basic (at least
version 4.0) to get the current versions of the DLLs the UI requires.
Older versions of the DLLs exist in the system directory and need to be
updated.
Answer: All of your program's output can be viewed in the console
window. To see it, choose "Console Window" from the options menu in the Etch
control window. Much of the output to the console
is from Etch and not from your program, so you must scroll down the
console until you find the program-generated outputs.
Question:How do I integrate DLLwatch output for a program that
I've run outside of the UI?
Answer: Since some programs may not be run from within the
UI, it may be necessary to run DLLwatch on your program directly from
the command line. Normally, the output from DLLwatch goes into
%WORKINGDIR%\dllwatch.output (%WORKINGDIR% is the current experiment folder),
so that it can be processed by the
UI. In order to do this same function manually, you will need to run
(from the same directory where you ran etch-run-me) the program
etch-dllwatch-filter (which can be found in the etch\bin
directory). After you've done this, you can then import the results
from the run using the "options|add explicitly loaded modules" menu
option in the Modules window.
Question: How do I create a standalone version of an etched application,
that is, a completely self-contained directory that
holds everything that the application needs to
execute?
Answer: Visual Etch assumes that an etched application will run in the context
of Visual Etch -- in particular, it assumes that certain DLLs are on
the path.
However, in some cases you may want to run an etched application
without making changes to the path or other parts of the environment,
in which case these assumptions will not hold. The simplest solution
to this problem is to collect all of the requisite DLLs into a single
directory. For example, if you've etched Netscape, you could create a
copy of the Netscape directory tree, and copy the etched application
(netscape-etch.exe) and all of the associated DLLs into the same
directory as netscape.exe in the copy of the Netscape directory tree..
Setting up the directory is reasonably straightforward:
Question:I wrote a tool and provided a routine in the tool
runtime DLL (say, ProgramBefore), and I implement the InstrumentProgram callback in the
tool instrumentation DLL. At runtime, though, my ProgramBefore function
is never called. What's going on?
Answer:
It is possible that you forgot to export your InstrumentProgram
callback. Etch can only make callbacks to functions that are exported
by the tool instrumentation DLL -- even if InstrumentProgram is
implemented, if it is not exported then Etch cannot call it.
Question:I wrote a tool that exports a runtime call WatchInstruction. But
when I run Etch, I get the message, "WatchInstruction not found". How
can this be?
Answer:
The most likely explanation is that your runtime function is a C++
function, and has had its name mangled. You should surround the
DllExport declaration of the function with an 'extern "C" { } '.
Answer: The ArgEtchedImageBase argument type can be used to uniquely
identify DLLs. This value is the actual base address of the etched
module at runtime, and so is guaranteed to be unique.
Or, you can pass constant strings to
runtime routines through the ArgConstString argument type, so you
could simply determine the module name (with GetEtchedModuleName) in
InstrumentInit, and then pass this string in on each call to
InsertCall(CheckInst,...). The ArgString argument type would not be a
good solution for this particular problem, because it would allocate
space for a new copy of the string for each call to
InsertCall(CheckInst,...).]
Question: Does Etch handle synchronization? For example, I have instrumented
each instruction to bump a counter. Does Etch worry about the fact that
different threads will be changing the value of this counter and hence
the existence of a race condition?
Answer: Etch does not handle this;
synchronization is entirely the responsibility of the
tool writer. Some of the example tools provided with Etch do
not take care of synchronization, and hence could report incorrect
results when used with a multi-threaded application.
Question: I followed all the directions for creating a new tool, and wrote my
own perl script called "mytool.pl". Now whenever I try to etch any
application from the UI, using *any* tool, I get the message "Etch process
exited prematurely". There is no output in the console window.
What did I do wrong?
Answer: You forgot to put a "1;" as the last line of mytool.pl. (This is
needed for Perl's mechanism for including files to work properly.)
System and program considerations for Etch
Question: Etch can sometimes take a very long time to run. How do I
know whether Etch is running or hung?
Question: Can multiple users run Etch from
the network?
Etch and DLLs
Question:How are etched files named?
Problems with your Etched programs
Question: I etched a program, and now it doesn't run. What do
I do next? Some specific problems
Installing and running Visual Etch -- The User Interface
Question: When I install Visual Basic, I get messages about
some registry keys or some files being used by other programs when
there isn't any other program running. What is causing this?
Problem: In the UI, I select a program, choose an action,
and click the execute action button, but I never see the output of
my program.
For example, for the count instruction references tool with winword, starting in
the Visual Etch experiment directory:
....EXPERIMENT...> COPY * <target>
....EXPERIMENT...> CD <target>
...target...>notepad etch.config (change first line of
etch.config to <target>\winword-etch.exe)
...target...> COPY %ETCHROOT%\apps\bin\countiref-rt.dll .
...target...> COPY %ETCHROOT%\apps\bin\utils.dll .
...target...> COPY %ETCHROOT%\bin\etchwrap.dll .
...target...> COPY %ETCHROOT%\bin\etchrt.dll .
Now the etched executable in the target directory should be able to run,
as long as the environment is the same as that required to run the original
application (e.g., if the application expects to find specific files in
specific subdirectories, those files and subdirectories must exist).
Developing Tools
Question: I need to associate a unique identifier with each module (each DLL
and EXE in the process) at runtime. For example, I've inserted a
'CheckInst' call before every instruction, and at runtime I want to
determine which module the instruction is in.
Does Etch provide any support for this?