Calculating Methods' Local Variable Sizes at Runtime

Discussion:

(too old to reply)

Nemo Caviani

2010-03-29 07:27:29 UTC

Hello All,

I am a newbie to ASM and I would appreciate your help with the following
matter.

I have load-time instrumented my bytecode. I keep all sorts of information
for my classes and methods to measure CPU time, memory usage, etc. Now, what
I need to do is to calculate the actual amount of data that is passed from
one method to another through the input parameters and also its return value
of the method.

This is in particular important for me as it enables me to measure the
actual amount of data that arrays and link lists take when they are passed
as arguments to a method or when they are returned from methods.

I have checked the visitLocalVariable method, but I am not sure how I can
relate the name and the description of a local variable to its actual data
size in the local stack memory, especially when it is not a basic type like
INT or BOOLEAN.

I also looked into visitFrame which seems to have more details about the
variables of a method. However, overwriting visitFrame results in a
VerifyError as it seems like I need to calculate the stack size for the
method manually, which I don't know how.

I highly appreciate any help from you guys.

thanks a lot,
Nemo

Eugene Kuleshov

2010-03-29 12:42:41 UTC

Permalink

So you want to calculate something at runtime before method call and
after the method call. For your scenario, primitive types are obvious
and for a non-primitive types the easiest would be to write a static
helper method, which you call for every parameter and non-primitive
method return value. All param and return types you can retrieve from
the called method signature and then generate code to call your static
method. You may have to create some temporary variables in order to get
the parameter values off the stack, but your transformation sould not
change state of the stack at the point original method is called.

Hope that helps.

regards,
Eugene

Post by Nemo Caviani
Hello All,
I am a newbie to ASM and I would appreciate your help with the
following matter.
I have load-time instrumented my bytecode. I keep all sorts of
information for my classes and methods to measure CPU time, memory
usage, etc. Now, what I need to do is to calculate the actual amount
of data that is passed from one method to another through the input
parameters and also its return value of the method.
This is in particular important for me as it enables me to measure the
actual amount of data that arrays and link lists take when they are
passed as arguments to a method or when they are returned from methods.
I have checked the visitLocalVariable method, but I am not sure how I
can relate the name and the description of a local variable to its
actual data size in the local stack memory, especially when it is not
a basic type like INT or BOOLEAN.
I also looked into visitFrame which seems to have more details about
the variables of a method. However, overwriting visitFrame results in
a VerifyError as it seems like I need to calculate the stack size for
the method manually, which I don't know how.
I highly appreciate any help from you guys.
thanks a lot,
Nemo

Nemo Caviani

2010-03-29 22:23:50 UTC

Permalink

Thanks Eugene for the reply.

yes, using visitLocalVariable I am able to identify the set of parameters
passed to a method. What I can't figure out is how to get the actual values
for these parameters off the stack to calculate their real sizes. I can
figure out their names and their types from the method signature, and for
the primitive types a lookup table can give me the size of their memory
usage. What I don't know, is how to get the actual size of non-primitive
parameters. If the nonprimitive type contains an array or a list, the length
of the array or the list also size becomes a critical factor in figuring out
the amount of space it uses.

so if you or anyone in the list can give me a clue on how to access the
actual value of a parameter from the local stack and calculate its size,
that would be a great help to me indeed.

thanks a lot,
Nemo

So you want to calculate something at runtime before method call and after
the method call. For your scenario, primitive types are obvious and for a
non-primitive types the easiest would be to write a static helper method,
which you call for every parameter and non-primitive method return value.
All param and return types you can retrieve from the called method signature
and then generate code to call your static method. You may have to create
some temporary variables in order to get the parameter values off the stack,
but your transformation sould not change state of the stack at the point
original method is called.
Hope that helps.
regards,
Eugene

Post by Nemo Caviani
Hello All,
I am a newbie to ASM and I would appreciate your help with the following
matter.
I have load-time instrumented my bytecode. I keep all sorts of information
for my classes and methods to measure CPU time, memory usage, etc. Now, what
I need to do is to calculate the actual amount of data that is passed from
one method to another through the input parameters and also its return value
of the method.
This is in particular important for me as it enables me to measure the
actual amount of data that arrays and link lists take when they are passed
as arguments to a method or when they are returned from methods.
I have checked the visitLocalVariable method, but I am not sure how I can
relate the name and the description of a local variable to its actual data
size in the local stack memory, especially when it is not a basic type like
INT or BOOLEAN.
I also looked into visitFrame which seems to have more details about the
variables of a method. However, overwriting visitFrame results in a
VerifyError as it seems like I need to calculate the stack size for the
method manually, which I don't know how.
I highly appreciate any help from you guys.
thanks a lot,
Nemo

--
OW2 mailing lists service home page: http://www.ow2.org/wws

Godmar Back

2010-03-29 22:41:02 UTC

Permalink

You need to implement it, like Eugene said.

Do a traversal of the object tree and sum up the sizes. It's expensive,
which is why this methods isn't commonly used for memory accounting.

See http://portal.acm.org/citation.cfm?id=1029888 and the papers it
references for more background.

- Godmar

Post by Nemo Caviani
Thanks Eugene for the reply.
yes, using visitLocalVariable I am able to identify the set of parameters
passed to a method. What I can't figure out is how to get the actual values
for these parameters off the stack to calculate their real sizes. I can
figure out their names and their types from the method signature, and for
the primitive types a lookup table can give me the size of their memory
usage. What I don't know, is how to get the actual size of non-primitive
parameters. If the nonprimitive type contains an array or a list, the length
of the array or the list also size becomes a critical factor in figuring out
the amount of space it uses.
so if you or anyone in the list can give me a clue on how to access the
actual value of a parameter from the local stack and calculate its size,
that would be a great help to me indeed.
thanks a lot,
Nemo

Post by Eugene Kuleshov
So you want to calculate something at runtime before method call and
after the method call. For your scenario, primitive types are obvious and
for a non-primitive types the easiest would be to write a static helper
method, which you call for every parameter and non-primitive method return
value. All param and return types you can retrieve from the called method
signature and then generate code to call your static method. You may have to
create some temporary variables in order to get the parameter values off the
stack, but your transformation sould not change state of the stack at the
point original method is called.
Hope that helps.
regards,
Eugene

Post by Nemo Caviani
Hello All,
I am a newbie to ASM and I would appreciate your help with the following
matter.
I have load-time instrumented my bytecode. I keep all sorts of
information for my classes and methods to measure CPU time, memory usage,
etc. Now, what I need to do is to calculate the actual amount of data that
is passed from one method to another through the input parameters and also
its return value of the method.
This is in particular important for me as it enables me to measure the
actual amount of data that arrays and link lists take when they are passed
as arguments to a method or when they are returned from methods.
I have checked the visitLocalVariable method, but I am not sure how I can
relate the name and the description of a local variable to its actual data
size in the local stack memory, especially when it is not a basic type like
INT or BOOLEAN.
I also looked into visitFrame which seems to have more details about the
variables of a method. However, overwriting visitFrame results in a
VerifyError as it seems like I need to calculate the stack size for the
method manually, which I don't know how.
I highly appreciate any help from you guys.
thanks a lot,
Nemo

--
OW2 mailing lists service home page: http://www.ow2.org/wws

Eugene Kuleshov

2010-03-30 00:00:37 UTC

Permalink

Post by Nemo Caviani
yes, using visitLocalVariable I am able to identify the set of
parameters passed to a method.

You don't need visitLocalVariable() at all

Post by Nemo Caviani
What I can't figure out is how to get the actual values for these
parameters off the stack to calculate their real sizes. I can figure
out their names and their types from the method signature, and for the
primitive types a lookup table can give me the size of their memory
usage. What I don't know, is how to get the actual size of
non-primitive parameters. If the nonprimitive type contains an array
or a list, the length of the array or the list also size becomes a
critical factor in figuring out the amount of space it uses.

You should really do your home work and at least search on Google. But
here is the reference for you, it is a bit outdated and as Godmar said
it is expensive to do it that way, but it is a start.
http://www.javaspecialists.eu/archive/Issue078.html

Post by Nemo Caviani
so if you or anyone in the list can give me a clue on how to access
the actual value of a parameter from the local stack and calculate its
size, that would be a great help to me indeed.

You will kinda have to learn some Java bytecode for that.
A naive, brute force method could be like the following:

Look at invokevirtual opcode. Documentation is here
http://java.sun.com/docs/books/jvms/second_edition/html/Instructions2.doc6.html#invokevirtual.Operation

At the visitMethodInsn(), the top of the stack will be like the following:
... , objectref, arg1, arg2, ... , argN
so argN is at the top of the stack (1 or 2 words depending on its
type), which you can get from the desc parameter of visitMethod(); then
2nd parameter and so on, the last would be the object reference the
method is called on.
So, you've got the stack figured out, now for each method parameter
you can create a new variable (for example, using LocalVariablesSorter
or one of its subclasses described in the ASM User guide book) and store
value off the stack into that variable using *store opcode corresponding
to parameter type.
After that for each non-primitive parameter you can load value from
the variables you created and call your static helper method to figure
out sizes.
Once you done with that, all you'll have to do is to load values from
those variables back into the stack, so invokevirtual instruction will
get the same stack as before your transformation.

There is some difference in handling invokevirtual, invokespecial,
invokestatic and invokeinterface instructions.

Obviously the above solution is suboptimal and can be heavily
optimized, e.g. to not create unnecessary variables in trivial cases
with 1 to 4 method parameters and for primitive parameters, e.g. using
stack manipulation bytecode instructions, but I will leave this exercise
for you.

regards,
Eugene

Nemo Caviani

2010-04-04 10:29:44 UTC

Permalink

Eugene, Godmar, and Jorgen,

Thank you very much for all your helps. All your helps where incredibly
useful to me. Eugene, in particular the second part of your email about
popping parameters from the local memory stack was exactly was I was
wondering about. The first part on calculating the variable sizes I had
already implemented but didn't quite know how to get the proper values for
the variable.

I need your last bit of advice to make sure that I am doing it the right
way. so, here is the code that I have written so far. The
PrefSignatureVisitor is a SignatureVisitor that returns the parsed list of
method signatures and computes the size of primitive datatypes. The elements
in the List returned by sigVisitor.getSigList() keep the type of input
parameters to the method except for the last one that keeps the return type
of the method. For the primitive data types, I only pop them up from the
queue whereas for the non primitive data types I invoke the static method
estimate to roughly measure their size. However, it throws exceptions when I
execute it. Could you let me know about what I am doing wrong?

sorry if the code is a bit messy. I am just trying to test it at the moment
and I will really appreciate your help

public void visitMethodInsn(int opcode, String owner, String name, String
desc) {

PrefSignatureVisitor sigVisitor =
parseAndMeasureBandwidth(opcode, name, desc);

if ((opcode == Opcodes.INVOKEVIRTUAL || opcode == Opcodes.INVOKEDYNAMIC ||
opcode== Opcodes.INVOKEINTERFACE)){

List<String> sigList = sigVisitor.getSigList();
String[] sigNameArray = new String[sigList.size()];
int[] localVarIndxArray = new int[sigList.size()];

for (int i = sigList.size() - 2; i >= 0; i--){

String type = sigList.get(i);
String varName = "tmp" + Integer.toString(i);
sigNameArray[i] = varName;

if (type.length() == 1){

switch(type.charAt(0)){
case 'Z':
localVarIndxArray[i] = _lvs.newLocal(Type.BOOLEAN_TYPE);
this.visitVarInsn(Opcodes.ISTORE, localVarIndxArray[i]);
break;
case 'C':
localVarIndxArray[i] = _lvs.newLocal(Type.CHAR_TYPE);
this.visitVarInsn(Opcodes.ISTORE, localVarIndxArray[i]);
break;
case 'B':
localVarIndxArray[i] = _lvs.newLocal(Type.BYTE_TYPE);
this.visitVarInsn(Opcodes.ISTORE, localVarIndxArray[i]);
break;
case 'S':
localVarIndxArray[i] = _lvs.newLocal(Type.SHORT_TYPE);
this.visitVarInsn(Opcodes.ISTORE, localVarIndxArray[i]);
break;
case 'I':
localVarIndxArray[i] = _lvs.newLocal(Type.INT_TYPE);
this.visitVarInsn(Opcodes.ISTORE, localVarIndxArray[i]);
break;
case 'F':
localVarIndxArray[i] = _lvs.newLocal(Type.FLOAT_TYPE);
this.visitVarInsn(Opcodes.FSTORE, localVarIndxArray[i]);
break;
case 'J':
localVarIndxArray[i] = _lvs.newLocal(Type.LONG_TYPE);
this.visitVarInsn(Opcodes.LSTORE, localVarIndxArray[i]);
break;
case 'D':
localVarIndxArray[i] = _lvs.newLocal(Type.DOUBLE_TYPE);
this.visitVarInsn(Opcodes.DSTORE, localVarIndxArray[i]);
break;
}

}else{
localVarIndxArray[i] = _lvs.newLocal(Type.getObjectType(sigList.get(i)));
this.visitVarInsn(Opcodes.ASTORE, localVarIndxArray[i]);
this.visitVarInsn(Opcodes.ALOAD, localVarIndxArray[i]);
this.visitMethodInsn(Opcodes.INVOKESTATIC,
"ca/ubc/magic/memory/MemoryCounter", "estimate", "(Ljava/lang/Object;)J");
}
}
for (int i = 0; i < sigList.size() - 1; i++){
String type = sigList.get(i);
if (type.length() == 1){
switch(type.charAt(0)){
case 'Z':
case 'C':
case 'B':
case 'S':
case 'I':
this.visitVarInsn(Opcodes.ILOAD, localVarIndxArray[i]);
break;
case 'F':
this.visitVarInsn(Opcodes.FLOAD, localVarIndxArray[i]);
break;
case 'J':
this.visitVarInsn(Opcodes.LLOAD, localVarIndxArray[i]);
break;
case 'D':
this.visitVarInsn(Opcodes.DLOAD, localVarIndxArray[i]);
break;
}
} else
this.visitVarInsn(Opcodes.ALOAD, localVarIndxArray[i]);
}
}

Post by Eugene Kuleshov

Post by Nemo Caviani
yes, using visitLocalVariable I am able to identify the set of parameters
passed to a method.

You don't need visitLocalVariable() at all
What I can't figure out is how to get the actual values for these

Post by Nemo Caviani
parameters off the stack to calculate their real sizes. I can figure out
their names and their types from the method signature, and for the primitive
types a lookup table can give me the size of their memory usage. What I
don't know, is how to get the actual size of non-primitive parameters. If
the nonprimitive type contains an array or a list, the length of the array
or the list also size becomes a critical factor in figuring out the amount
of space it uses.

You should really do your home work and at least search on Google. But
here is the reference for you, it is a bit outdated and as Godmar said it
is expensive to do it that way, but it is a start.
http://www.javaspecialists.eu/archive/Issue078.html
so if you or anyone in the list can give me a clue on how to access the

Post by Nemo Caviani
actual value of a parameter from the local stack and calculate its size,
that would be a great help to me indeed.

You will kinda have to learn some Java bytecode for that.
Look at invokevirtual opcode. Documentation is here
http://java.sun.com/docs/books/jvms/second_edition/html/Instructions2.doc6.html#invokevirtual.Operation
... , objectref, arg1, arg2, ... , argN
so argN is at the top of the stack (1 or 2 words depending on its type),
which you can get from the desc parameter of visitMethod(); then 2nd
parameter and so on, the last would be the object reference the method is
called on.
So, you've got the stack figured out, now for each method parameter you
can create a new variable (for example, using LocalVariablesSorter or one of
its subclasses described in the ASM User guide book) and store value off the
stack into that variable using *store opcode corresponding to parameter
type.
After that for each non-primitive parameter you can load value from the
variables you created and call your static helper method to figure out
sizes.
Once you done with that, all you'll have to do is to load values from
those variables back into the stack, so invokevirtual instruction will get
the same stack as before your transformation.
There is some difference in handling invokevirtual, invokespecial,
invokestatic and invokeinterface instructions.
Obviously the above solution is suboptimal and can be heavily optimized,
e.g. to not create unnecessary variables in trivial cases with 1 to 4 method
parameters and for primitive parameters, e.g. using stack manipulation
bytecode instructions, but I will leave this exercise for you.
regards,
Eugene
--
OW2 mailing lists service home page: http://www.ow2.org/wws

Nemo Caviani

2010-04-05 09:36:23 UTC

Permalink

Hi Everyone,

please forget my previous email. I think I got it figured out! :-)

thanks for all the hints,
Nemo

Post by Nemo Caviani
Eugene, Godmar, and Jorgen,
Thank you very much for all your helps. All your helps where incredibly
useful to me. Eugene, in particular the second part of your email about
popping parameters from the local memory stack was exactly was I was
wondering about. The first part on calculating the variable sizes I had
already implemented but didn't quite know how to get the proper values for
the variable.
I need your last bit of advice to make sure that I am doing it the right
way. so, here is the code that I have written so far. The
PrefSignatureVisitor is a SignatureVisitor that returns the parsed list of
method signatures and computes the size of primitive datatypes. The elements
in the List returned by sigVisitor.getSigList() keep the type of input
parameters to the method except for the last one that keeps the return type
of the method. For the primitive data types, I only pop them up from the
queue whereas for the non primitive data types I invoke the static method
estimate to roughly measure their size. However, it throws exceptions when I
execute it. Could you let me know about what I am doing wrong?
sorry if the code is a bit messy. I am just trying to test it at the moment
and I will really appreciate your help
public void visitMethodInsn(int opcode, String owner, String name, String
desc) {
PrefSignatureVisitor sigVisitor =
parseAndMeasureBandwidth(opcode, name, desc);
if ((opcode == Opcodes.INVOKEVIRTUAL || opcode == Opcodes.INVOKEDYNAMIC ||
opcode== Opcodes.INVOKEINTERFACE)){
List<String> sigList = sigVisitor.getSigList();
String[] sigNameArray = new String[sigList.size()];
int[] localVarIndxArray = new int[sigList.size()];
for (int i = sigList.size() - 2; i >= 0; i--){
String type = sigList.get(i);
String varName = "tmp" + Integer.toString(i);
sigNameArray[i] = varName;
if (type.length() == 1){
switch(type.charAt(0)){
localVarIndxArray[i] = _lvs.newLocal(Type.BOOLEAN_TYPE);
this.visitVarInsn(Opcodes.ISTORE, localVarIndxArray[i]);
break;
localVarIndxArray[i] = _lvs.newLocal(Type.CHAR_TYPE);
this.visitVarInsn(Opcodes.ISTORE, localVarIndxArray[i]);
break;
localVarIndxArray[i] = _lvs.newLocal(Type.BYTE_TYPE);
this.visitVarInsn(Opcodes.ISTORE, localVarIndxArray[i]);
break;
localVarIndxArray[i] = _lvs.newLocal(Type.SHORT_TYPE);
this.visitVarInsn(Opcodes.ISTORE, localVarIndxArray[i]);
break;
localVarIndxArray[i] = _lvs.newLocal(Type.INT_TYPE);
this.visitVarInsn(Opcodes.ISTORE, localVarIndxArray[i]);
break;
localVarIndxArray[i] = _lvs.newLocal(Type.FLOAT_TYPE);
this.visitVarInsn(Opcodes.FSTORE, localVarIndxArray[i]);
break;
localVarIndxArray[i] = _lvs.newLocal(Type.LONG_TYPE);
this.visitVarInsn(Opcodes.LSTORE, localVarIndxArray[i]);
break;
localVarIndxArray[i] = _lvs.newLocal(Type.DOUBLE_TYPE);
this.visitVarInsn(Opcodes.DSTORE, localVarIndxArray[i]);
break;
}
}else{
localVarIndxArray[i] = _lvs.newLocal(Type.getObjectType(sigList.get(i)));
this.visitVarInsn(Opcodes.ASTORE, localVarIndxArray[i]);
this.visitVarInsn(Opcodes.ALOAD, localVarIndxArray[i]);
this.visitMethodInsn(Opcodes.INVOKESTATIC,
"ca/ubc/magic/memory/MemoryCounter", "estimate", "(Ljava/lang/Object;)J");
}
}
for (int i = 0; i < sigList.size() - 1; i++){
String type = sigList.get(i);
if (type.length() == 1){
switch(type.charAt(0)){
this.visitVarInsn(Opcodes.ILOAD, localVarIndxArray[i]);
break;
this.visitVarInsn(Opcodes.FLOAD, localVarIndxArray[i]);
break;
this.visitVarInsn(Opcodes.LLOAD, localVarIndxArray[i]);
break;
this.visitVarInsn(Opcodes.DLOAD, localVarIndxArray[i]);
break;
}
} else
this.visitVarInsn(Opcodes.ALOAD, localVarIndxArray[i]);
}
}

Post by Eugene Kuleshov

Post by Nemo Caviani
yes, using visitLocalVariable I am able to identify the set of parameters
passed to a method.

You don't need visitLocalVariable() at all
What I can't figure out is how to get the actual values for these

You should really do your home work and at least search on Google. But
here is the reference for you, it is a bit outdated and as Godmar said it
is expensive to do it that way, but it is a start.
http://www.javaspecialists.eu/archive/Issue078.html
so if you or anyone in the list can give me a clue on how to access the

Post by Nemo Caviani
actual value of a parameter from the local stack and calculate its size,
that would be a great help to me indeed.