GDB Commands

Posted on by
Help Commands
help
command		 Get help on a certain command
apropos
keyword		 Search help for a particular
keyword
Starting and Quitting
gdb [-tui]
[-c core] [exename]		 (Unix Command)
Start gdb on an executable or standalone; specify
“-tui” to start the TUI GUI; specify “-c” with a corefile name to see
where a crash occurred
run [arg1] [arg2]
[…]		 Run the currently loaded program with
the given command line arguments
quit Exit the debugger
file
exename		 Load an executable file by
name
Breakpoints and Watchpoints
break
location		 Set a breakpoint at a location,
line number, or file (e.g. “main”, “5”, or “hello.c:23”)
watch
expression		 Break when a variable is
written to
rwatch
expression		 Break when a variable is
read from
awatch
expression		 Break when a variable is
written to or read from
info break Display breakpoint and watchpoint information and numbers
info watch Same as info break
clear
location		 Clear a breakpoint from a
location
delete num Delete a breakpoint or watchpoint by number
Stepping and Running
next Run to the next line of this function
step Step into the function on this line, if possible
stepi Step a single assembly instruction
continue Keep running from here
CTRL-C Stop running, wherever you are
finish Run until the end of the current function
advance
location		 Advance to a location,
line number, or file (e.g. “somefunction”, “5”, or “hello.c:23”)
jump location Just like continue, except jump to
a particular location first.
Examining and Modifying Variables
display
expression		 Display the value of a
variable or expression every step of the program—the expression
must make sense in the current scope
info display Show a list of expressions currently being displayed and
their numbers
undisplay
num		 Stop showing an expression
identified by its number (see info display)
print
expression		 Print the value of a variable
or expression
printf formatstr expressionlist Do some
formatted output with printf() e.g. printf "i = %d, p = %s\n", i, p
set variable
expression		 Set a variable to value, e.g.
set variable x=20
set
(expression)		 Works like set variable
Window Commands
info win Shows current window info
focus
winname		 Set focus to a particular window
bby name (“SRC”, “CMD”, “ASM”, or “REG”) or by position (“next” or
“prev”)
fs Alias for focus
layout type Set the window layout (“src”, “asm”, “split”, or “reg”)
tui reg type Set the register window layout (“general”, “float”,
“system”, or “next”)
winheight val Set the window height (either an absolute value, or a
relative value prefaced with “+” or “-“)
wh Alias for winheight
set disassembly-flavor flavor Set the look-and-feel of the disassembly. On Intel
machines, valid flavors are intel and att
Misc Commands
RETURN Hit RETURN to repeat the last command
backtrace Show the current stack
bt Alias for backtrace
attach pid Attach to an already-running process by its PID
info registers Dump integer registers to screen
info all-registers Dump all registers to screen

 

Wildcards when git adding files

Posted on by

From Git documentation:

Adds content from all *.txt files under Documentation directory and its subdirectories:

$ git add Documentation/\*.txt

Note that the asterisk * is quoted from the shell in this example; this lets the command include the files from subdirectories of Documentation/ directory.

Considers adding content from all git-*.sh scripts:

$ git add git-*.sh

Because this example lets the shell expand the asterisk (i.e. you are listing the files explicitly), it does not consider subdir/git-foo.sh.

Fixing unreadable tooltips in Eclipse on Ubuntu

Posted on by

In Eclipse on ubuntu 10.04, the tooltip that appears when hovering with the mouse over some code uses black font on black background, which is unreadable. To fix it, do following workaround:

right click on desktop,
click change desktop background,
click theme,
click customize,
click colors
As for tooltips, change background colour back to #F6F88B, and foreground colour to#000000.

This makes tooltips appear black fonts on yellow background.

Bash Config Files

Posted on by

Here’s how Bash executes startup files (by Hermann Heimhardt on October 7, 2001)

For Login shells (subject to the -noprofile option):

On logging in:
If `/etc/profile’ exists, then source it.

If `~/.bash_profile’ exists, then source it,
else if `~/.bash_login’ exists, then source it,
else if `~/.profile’ exists, then source it.

On logging out:
If `~/.bash_logout’ exists, source it.

For non-login interactive shells (subject to the -norc and -rcfile options):
On starting up:
If `~/.bashrc’ exists, then source it.

For non-interactive shells:
On starting up:
If the environment variable `ENV’ is non-null, expand the variable and source the file named by the value. If Bash is not started in Posix mode, it looks for `BASH_ENV’ before `ENV’.

So, typically, your `~/.bash_profile’ contains the line
`if [ -f `~/.bashrc’ ]; then source `~/.bashrc’; fi’ after (or before) any login specific initializations.

If Bash is invoked as `sh’, it tries to mimic the behavior of `sh’ as closely as possible. For a login shell, it attempts to source only `/etc/profile’ and `~/.profile’, in that order. The `-noprofile’ option may still be used to disable this behavior. A shell invoked as `sh’ does not attempt to source any other startup files.

When Bash is started in POSIX mode, as with the `-posix’ command line option, it follows the Posix 1003.2 standard for startup files. In this mode, the `ENV’ variable is expanded and that file sourced; no other startup files are read.

.inputrc (readline)– Although the Readline library comes with a set of Emacs-like key bindings installed by default, it is possible that you would like to use a different set of keybindings. You can customize programs that use Readline by putting commands in an “init” file in your home directory. The name of this file is taken from the value of the shell variable `INPUTRC’. If that variable is unset, the default is `~/.inputrc’.

When a program which uses the Readline library starts up, the init file is read, and the key bindings are set.

In addition, the `C-x C-r’ command re-reads this init file, thus incorporating any changes that you might have made to it.

Java Bytecode

Posted on by

Java bytecode instruction listings

Mnemonic Opcode (in hex) Other bytes Stack [before]→[after] Description
aaload 32 arrayref, index → value loads onto the stack a reference from an array
aastore 53 arrayref, index, value → stores into a reference in an array
aconst_null 01 → null pushes a null reference onto the stack
aload 19 1: index → objectref loads a reference onto the stack from a local variable #index
aload_0 2a → objectref loads a reference onto the stack from local variable 0
aload_1 2b → objectref loads a reference onto the stack from local variable 1
aload_2 2c → objectref loads a reference onto the stack from local variable 2
aload_3 2d → objectref loads a reference onto the stack from local variable 3
anewarray bd 2: indexbyte1, indexbyte2 count → arrayref creates a new array of references of length count and component type identified by the class reference index (indexbyte1 << 8 + indexbyte2) in the constant pool
areturn b0 objectref → [empty] returns a reference from a method
arraylength be arrayref → length gets the length of an array
astore 3a 1: index objectref → stores a reference into a local variable #index
astore_0 4b objectref → stores a reference into local variable 0
astore_1 4c objectref → stores a reference into local variable 1
astore_2 4d objectref → stores a reference into local variable 2
astore_3 4e objectref → stores a reference into local variable 3
athrow bf objectref → [empty], objectref throws an error or exception (notice that the rest of the stack is cleared, leaving only a reference to the Throwable)
baload 33 arrayref, index → value loads a byte or Boolean value from an array
bastore 54 arrayref, index, value → stores a byte or Boolean value into an array
bipush 10 1: byte → value pushes a byte onto the stack as an integer value
caload 34 arrayref, index → value loads a char from an array
castore 55 arrayref, index, value → stores a char into an array
checkcast c0 2: indexbyte1, indexbyte2 objectref → objectref checks whether an objectref is of a certain type, the class reference of which is in the constant pool at index (indexbyte1 << 8 + indexbyte2)
d2f 90 value → result converts a double to a float
d2i 8e value → result converts a double to an int
d2l 8f value → result converts a double to a long
dadd 63 value1, value2 → result adds two doubles
daload 31 arrayref, index → value loads a double from an array
dastore 52 arrayref, index, value → stores a double into an array
dcmpg 98 value1, value2 → result compares two doubles
dcmpl 97 value1, value2 → result compares two doubles
dconst_0 0e → 0.0 pushes the constant 0.0 onto the stack
dconst_1 0f → 1.0 pushes the constant 1.0 onto the stack
ddiv 6f value1, value2 → result divides two doubles
dload 18 1: index → value loads a double value from a local variable #index
dload_0 26 → value loads a double from local variable 0
dload_1 27 → value loads a double from local variable 1
dload_2 28 → value loads a double from local variable 2
dload_3 29 → value loads a double from local variable 3
dmul 6b value1, value2 → result multiplies two doubles
dneg 77 value → result negates a double
drem 73 value1, value2 → result gets the remainder from a division between two doubles
dreturn af value → [empty] returns a double from a method
dstore 39 1: index value → stores a double value into a local variable #index
dstore_0 47 value → stores a double into local variable 0
dstore_1 48 value → stores a double into local variable 1
dstore_2 49 value → stores a double into local variable 2
dstore_3 4a value → stores a double into local variable 3
dsub 67 value1, value2 → result subtracts a double from another
dup 59 value → value, value duplicates the value on top of the stack
dup_x1 5a value2, value1 → value1, value2, value1 inserts a copy of the top value into the stack two values from the top
dup_x2 5b value3, value2, value1 → value1, value3, value2, value1 inserts a copy of the top value into the stack two (if value2 is double or long it takes up the entry of value3, too) or three values (if value2 is neither double nor long) from the top
dup2 5c {value2, value1} → {value2, value1}, {value2, value1} duplicate top two stack words (two values, if value1 is not double nor long; a single value, if value1 is double or long)
dup2_x1 5d value3, {value2, value1} → {value2, value1}, value3, {value2, value1} duplicate two words and insert beneath third word (see explanation above)
dup2_x2 5e {value4, value3}, {value2, value1} → {value2, value1}, {value4, value3}, {value2, value1} duplicate two words and insert beneath fourth word
f2d 8d value → result converts a float to a double
f2i 8b value → result converts a float to an int
f2l 8c value → result converts a float to a long
fadd 62 value1, value2 → result adds two floats
faload 30 arrayref, index → value loads a float from an array
fastore 51 arrayref, index, value → stores a float in an array
fcmpg 96 value1, value2 → result compares two floats
fcmpl 95 value1, value2 → result compares two floats
fconst_0 0b → 0.0f pushes 0.0f on the stack
fconst_1 0c → 1.0f pushes 1.0f on the stack
fconst_2 0d → 2.0f pushes 2.0f on the stack
fdiv 6e value1, value2 → result divides two floats
fload 17 1: index → value loads a float value from a local variable #index
fload_0 22 → value loads a float value from local variable 0
fload_1 23 → value loads a float value from local variable 1
fload_2 24 → value loads a float value from local variable 2
fload_3 25 → value loads a float value from local variable 3
fmul 6a value1, value2 → result multiplies two floats
fneg 76 value → result negates a float
frem 72 value1, value2 → result gets the remainder from a division between two floats
freturn ae value → [empty] returns a float
fstore 38 1: index value → stores a float value into a local variable #index
fstore_0 43 value → stores a float value into local variable 0
fstore_1 44 value → stores a float value into local variable 1
fstore_2 45 value → stores a float value into local variable 2
fstore_3 46 value → stores a float value into local variable 3
fsub 66 value1, value2 → result subtracts two floats
getfield b4 2: index1, index2 objectref → value gets a field value of an object objectref, where the field is identified by field reference in the constant pool index (index1 << 8 + index2)
getstatic b2 2: index1, index2 → value gets a static field value of a class, where the field is identified by field reference in the constant pool index (index1 << 8 + index2)
goto a7 2: branchbyte1, branchbyte2 [no change] goes to another instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
goto_w c8 4: branchbyte1, branchbyte2, branchbyte3, branchbyte4 [no change] goes to another instruction at branchoffset (signed int constructed from unsigned bytes branchbyte1 << 24 + branchbyte2 << 16 + branchbyte3 << 8 + branchbyte4)
i2b 91 value → result converts an int into a byte
i2c 92 value → result converts an int into a character
i2d 87 value → result converts an int into a double
i2f 86 value → result converts an int into a float
i2l 85 value → result converts an int into a long
i2s 93 value → result converts an int into a short
iadd 60 value1, value2 → result adds two ints together
iaload 2e arrayref, index → value loads an int from an array
iand 7e value1, value2 → result performs a bitwise and on two integers
iastore 4f arrayref, index, value → stores an int into an array
iconst_m1 02 → -1 loads the int value -1 onto the stack
iconst_0 03 → 0 loads the int value 0 onto the stack
iconst_1 04 → 1 loads the int value 1 onto the stack
iconst_2 05 → 2 loads the int value 2 onto the stack
iconst_3 06 → 3 loads the int value 3 onto the stack
iconst_4 07 → 4 loads the int value 4 onto the stack
iconst_5 08 → 5 loads the int value 5 onto the stack
idiv 6c value1, value2 → result divides two integers
if_acmpeq a5 2: branchbyte1, branchbyte2 value1, value2 → if references are equal, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
if_acmpne a6 2: branchbyte1, branchbyte2 value1, value2 → if references are not equal, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
if_icmpeq 9f 2: branchbyte1, branchbyte2 value1, value2 → if ints are equal, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
if_icmpne a0 2: branchbyte1, branchbyte2 value1, value2 → if ints are not equal, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
if_icmplt a1 2: branchbyte1, branchbyte2 value1, value2 → if value1 is less than value2, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
if_icmpge a2 2: branchbyte1, branchbyte2 value1, value2 → if value1 is greater than or equal to value2, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
if_icmpgt a3 2: branchbyte1, branchbyte2 value1, value2 → if value1 is greater than value2, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
if_icmple a4 2: branchbyte1, branchbyte2 value1, value2 → if value1 is less than or equal to value2, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
ifeq 99 2: branchbyte1, branchbyte2 value → if value is 0, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
ifne 9a 2: branchbyte1, branchbyte2 value → if value is not 0, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
iflt 9b 2: branchbyte1, branchbyte2 value → if value is less than 0, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
ifge 9c 2: branchbyte1, branchbyte2 value → if value is greater than or equal to 0, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
ifgt 9d 2: branchbyte1, branchbyte2 value → if value is greater than 0, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
ifle 9e 2: branchbyte1, branchbyte2 value → if value is less than or equal to 0, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
ifnonnull c7 2: branchbyte1, branchbyte2 value → if value is not null, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
ifnull c6 2: branchbyte1, branchbyte2 value → if value is null, branch to instruction at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2)
iinc 84 2: index, const [No change] increment local variable #index by signed byte const
iload 15 1: index → value loads an int value from a local variable #index
iload_0 1a → value loads an int value from local variable 0
iload_1 1b → value loads an int value from local variable 1
iload_2 1c → value loads an int value from local variable 2
iload_3 1d → value loads an int value from local variable 3
imul 68 value1, value2 → result multiply two integers
ineg 74 value → result negate int
instanceof c1 2: indexbyte1, indexbyte2 objectref → result determines if an object objectref is of a given type, identified by class reference index in constant pool (indexbyte1 << 8 + indexbyte2)
invokeinterface b9 4: indexbyte1, indexbyte2, count, 0 objectref, [arg1, arg2, …] → invokes an interface method on object objectref, where the interface method is identified by method reference index in constant pool (indexbyte1 << 8 + indexbyte2)
invokespecial b7 2: indexbyte1, indexbyte2 objectref, [arg1, arg2, …] → invoke instance method on object objectref, where the method is identified by method reference index in constant pool (indexbyte1 << 8 + indexbyte2)
invokestatic b8 2: indexbyte1, indexbyte2 [arg1, arg2, …] → invoke a static method, where the method is identified by method reference index in constant pool (indexbyte1 << 8 + indexbyte2)
invokevirtual b6 2: indexbyte1, indexbyte2 objectref, [arg1, arg2, …] → invoke virtual method on object objectref, where the method is identified by method reference index in constant pool (indexbyte1 << 8 + indexbyte2)
ior 80 value1, value2 → result bitwise int or
irem 70 value1, value2 → result logical int remainder
ireturn ac value → [empty] returns an integer from a method
ishl 78 value1, value2 → result int shift left
ishr 7a value1, value2 → result int arithmetic shift right
istore 36 1: index value → store int value into variable #index
istore_0 3b value → store int value into variable 0
istore_1 3c value → store int value into variable 1
istore_2 3d value → store int value into variable 2
istore_3 3e value → store int value into variable 3
isub 64 value1, value2 → result int subtract
iushr 7c value1, value2 → result int logical shift right
ixor 82 value1, value2 → result int xor
jsr a8 2: branchbyte1, branchbyte2 → address jump to subroutine at branchoffset (signed short constructed from unsigned bytes branchbyte1 << 8 + branchbyte2) and place the return address on the stack
jsr_w c9 4: branchbyte1, branchbyte2, branchbyte3, branchbyte4 → address jump to subroutine at branchoffset (signed int constructed from unsigned bytes branchbyte1 << 24 + branchbyte2 << 16 + branchbyte3 << 8 + branchbyte4) and place the return address on the stack
l2d 8a value → result converts a long to a double
l2f 89 value → result converts a long to a float
l2i 88 value → result converts a long to a int
ladd 61 value1, value2 → result add two longs
laload 2f arrayref, index → value load a long from an array
land 7f value1, value2 → result bitwise and of two longs
lastore 50 arrayref, index, value → store a long to an array
lcmp 94 value1, value2 → result compares two longs values
lconst_0 09 → 0L pushes the long 0 onto the stack
lconst_1 0a → 1L pushes the long 1 onto the stack
ldc 12 1: indextd>→ valuepushes a constant #index from a constant pool (String, int or float) onto the stack
ldc_w 13 2: indexbyte1, indexbyte2 → value pushes a constant #index from a constant pool (String, int or float) onto the stack (wide index is constructed as indexbyte1 << 8 + indexbyte2)
ldc2_w 14 2: indexbyte1, indexbyte2 → value pushes a constant #index from a constant pool (double or long) onto the stack (wide index is constructed as indexbyte1 << 8 + indexbyte2)
ldiv 6d value1, value2 → result divide two longs
lload 16 1: index → value load a long value from a local variable #index
lload_0 1e → value load a long value from a local variable 0
lload_1 1f → value load a long value from a local variable 1
lload_2 20 → value load a long value from a local variable 2
lload_3 21 → value load a long value from a local variable 3
lmul 69 value1, value2 → result multiplies two longs
lneg 75 value → result negates a long
lookupswitch ab 4+: <0-3 bytes padding>, defaultbyte1, defaultbyte2, defaultbyte3, defaultbyte4, npairs1, npairs2, npairs3, npairs4, match-offset pairs… key → a target address is looked up from a table using a key and execution continues from the instruction at that address
lor 81 value1, value2 → result bitwise or of two longs
lrem 71 value1, value2 → result remainder of division of two longs
lreturn ad value → [empty] returns a long value
lshl 79 value1, value2 → result bitwise shift left of a long value1 by value2 positions
lshr 7b value1, value2 → result bitwise shift right of a long value1 by value2 positions
lstore 37 1: index value → store a long value in a local variable #index
lstore_0 3f value → store a long value in a local variable 0
lstore_1 40 value → store a long value in a local variable 1
lstore_2 41 value → store a long value in a local variable 2
lstore_3 42 value → store a long value in a local variable 3
lsub 65 value1, value2 → result subtract two longs
lushr 7d value1, value2 → result bitwise shift right of a long value1 by value2 positions, unsigned
lxor 83 value1, value2 → result bitwise exclusive or of two longs
monitorenter c2 objectref → enter monitor for object (“grab the lock” – start of synchronized() section)
monitorexit c3 objectref → exit monitor for object (“release the lock” – end of synchronized() section)
multianewarray c5 3: indexbyte1, indexbyte2, dimensions count1, [count2,…] → arrayref create a new array of dimensions dimensions with elements of type identified by class reference in constant pool index (indexbyte1 << 8 + indexbyte2); the sizes of each dimension is identified by count1, [count2, etc.]
new bb 2: indexbyte1, indexbyte2 → objectref creates new object of type identified by class reference in constant pool index (indexbyte1 << 8 + indexbyte2)
newarray bc 1: atype count → arrayref creates new array with count elements of primitive type identified by atype
nop 00 [No change] performs no operation
pop 57 value → discards the top value on the stack
pop2 58 {value2, value1} → discards the top two values on the stack (or one value, if it is a double or long)
putfield b5 2: indexbyte1, indexbyte2 objectref, value → set field to value in an object objectref, where the field is identified by a field reference index in constant pool (indexbyte1 << 8 + indexbyte2)
putstatic b3 2: indexbyte1, indexbyte2 value → set static field to value in a class, where the field is identified by a field reference index in constant pool (indexbyte1 << 8 + indexbyte2)
ret a9 1: index [No change] continue execution from address taken from a local variable #index (the asymmetry with jsr is intentional)
return b1 → [empty] return void from method
saload 35 arrayref, index → value load short from array
sastore 56 arrayref, index, value → store short to array
sipush 11 2: byte1, byte2 → value pushes a short onto the stack
swap 5f value2, value1 → value1, value2 swaps two top words on the stack (note that value1 and value2 must not be double or long)
tableswitch aa 4+: [0-3 bytes padding], defaultbyte1, defaultbyte2, defaultbyte3, defaultbyte4, lowbyte1, lowbyte2, lowbyte3, lowbyte4, highbyte1, highbyte2, highbyte3, highbyte4, jump offsets… index → continue execution from an address in the table at offset index
wide c4 3/5: opcode, indexbyte1, indexbyte2or

iinc, indexbyte1, indexbyte2, countbyte1, countbyte2

 [same as for corresponding instructions] execute opcode, where opcode is either iload, fload, aload, lload, dload, istore, fstore, astore, lstore, dstore, or ret, but assume the index is 16 bit; or execute iinc, where the index is 16 bits and the constant to increment by is a signed 16 bit short
breakpoint ca reserved for breakpoints in Java debuggers; should not appear in any class file
impdep1 fe reserved for implementation-dependent operations within debuggers; should not appear in any class file
impdep2 ff reserved for implementation-dependent operations within debuggers; should not appear in any class file
(no name) cb-fd these values are currently unassigned for opcodes and are reserved for future use
xxxunusedxxx ba this opcode is reserved “for historical reasons”

from Wikipedia

Garbage Collection for String

Posted on by

In Java you can create String from byte array, char array, another string, from StringBuffer or from StringBuilder. Java String class provides constructor for all of these. Since String is an immutable class every time a new String is created and older one is discarded which creates lots of temporary garbage in heap. If String are created using String literal they remain in String pool. For example,

String str="my string";

The string “my string” is ripped out by the compiler, replaced with a symbol and the value moved to the literal pool. The compiler will replace all instances of “my string” with this single reference to the value in the pool, a simple way to optimize memory use. These values are never garbage collected. Instances of String created at runtime are subject to garbage collection just like every other object.

String str= "my string";   
for (int i = 0; i < 10; i++){
   System.out.println(str+ i);
}

The above code creates 10 String instances eligible for garbage collection. Concatenation with + actually gets expanded into creating a new StringBuffer (maybe StringBuilder in new JDK’s) instance, then append() calls for each +, then finally a call to toString() which creates a new String instance.

The literal pool cannot get “full”. The compiler creates it by pulling the literals out of your code, so it is a set size when you start your program. It neither grows nor shrinks. The heap can get full and there are steps you can take to mitigate that.

The constant pool is a region in a .class file for all constants referenced in the class definition, not just strings. As for the String intern pool, that’s shared with String’s intern() method – which means it certainly can grow. It can also grow if you keep loading new classes into the JVM. It’s also possible for items in the intern pool to be garbage collected, but that’s fairly unusual, at least for literals, since such object also have other references from the class definition. You can think of the intern pool as a WeakHashMap – by itself, the intern pool will not prevent GC of its contents. But if there are any other references to a String in the pool, those can prevent GC. And it turns out that such references are maintained by the class in memory, after being loaded by a ClassLoader. So the only way for a literal to get GC’d is for the class to get unloaded. Which in turn requires that the ClassLoader that loaded it is also eligible for GC. Which doesn’t happen often in beginner code, but it’s not unusual in something like an app server. Bottom line, items in the intern pool can be GC’d, but it’s complex and unusual.

Latent Semantic Indexing

Posted on by
Introduction

Traditional search engines generate relevant documents by matching terms in a query. However, it is possible to miss or wrongly return relevant documents without knowing the true semantic meaning of documents. Some documents containing synonyms to a keyword may be semantically closely related to it even though these documents don’t match the keyword. Some documents in the search results may be not related to the keyword because a word may have multiple meanings even though they match the keyword exactly .Latent semantic indexing (LSI) overcome this problem by examining the document pool as a whole to find the relationships between the terms and concepts contained in an unstructured collection of text. It is based on the principle that words that are used in the same contexts tend to have similar meanings. Being able to capture the latent semantic structure in a collection of text, LSI helps to establish the relations between terms in similar context. A search engine undergoing LSI can return results that are conceptually similar in meaning to query terms even if the returned documents contains no same keywords as in the query.

Approach

Regular keyword searches approach a document collection by matching a keyword exactly. The search engine looks through each document to match the keywords contained in a query. Then it returns the documents contain the keywords by ranking them based on some algorithms. This kind of search leaves out documents without the keywords in the query yet having similar content, which should have been considered to be relevant documents to queries.

Wikipedia difines Latent Semantic Indexing as following:

Latent Semantic Indexing is an indexing and retrieval method that uses a mathematical technique called Singular value decomposition (SVD) to identify patterns in the relationships between the terms and concepts contained in an unstructured collection of text. [ Source: http://en.wikipedia.org/wiki/Latent_semantic_indexing ]

Basically, LSI follows the principle that words that are used in the same contexts tend to have similar meanings. This method works just like a human being when she tries to identify the content of a document by examining the words in it. In the indexing process of the search engine, besides indexing which keywords a document contains, LSI analyzes the collection of documents as a whole, to caputre other documents contain some of those similar words. If documents share many words, we consider them to be semantically close. Otherwise, we decide that they are semantically distant.

When we search an LSI-indexed collection of text, the search engine examines similarity values it has calculated for every content word, and returns the documents that it thinks best match the query. Since two documents may be semantically related to each other even if they do not have any specific keyword in common, LSI does not require an exact match to return useful results. Where a plain keyword search will fail if there is no exact match, LSI will often return relevant documents that don’t contain the keyword at all. [ Source: http://javelina.cet.middlebury.edu/l…definition.htm ]

Application

LSI is often used to overcome two issues in search engines: multiple words with similar meanings and words with multiple meanings. With the help of LSI, search engines can catch the relations between terms and conceptual content of a collection of documents . Amit Singhal, Google’s head of search quality, wrote:

It is critical that we understand what our users are looking for (beyond just the few words in their query). We have made several notable advances in this area including a best-in-class spelling suggestion system, an advanced synonyms system, and a very strong concept analysis system. [ Source: http://googleblog.blogspot.com/2008/07/technologies-behind-google-ranking.html ]

However, LSI is different from search based on synonyms. For example, search Google for “~available” and you will get a list of results with term “free” ranking as top two related documents. That is because the search engine of Google generates results with synonyms of “available” instead of just matching it. But this is not LSI since the pair of synonyms (“available” and “free” ) are known from dictionary. In a LSI-enabled search, we’ll get results related to query terms in a context. Try to search for “apple computer” and pay attention to the displayed results in Google AdSense. We’ll find ads with “mac laptop” appear in the top results. This is because “mac laptop” and “apple computer” often appear in the same context in a large number of documents.

Automated document categorization can easily be done with LSI.  Document categorization is the process that documents are assigned to some categories based on their similarity to the conceptual content of the categories. First, some example documents are analyzed by LSI to create the conceptual content for each category. Then the conceptual content contained in the raw documents are compared to the concepts contained in the example documents so that each document is assigned to one or more categories based on their similarity to the conceptual content of those categories.

LSI can also be used to dynamically retrieve similar documents according to their conceptual content. We group documents based on their similarity in conceptual content without knowing the actual meaning of each word in a collection of text. Google Similar Pages provide similar documents search but is still in beta state. Users checking a page returned by Google search may be interested in other similar pages.  Google Similar Pages can help to find more pages about a topic you’re researching.

Limitations

Scalability and performance were two big challenges to LSI in early years when it was applied in practical use. Due to the high complexity of its computing process, LSI requires a lot of computing power as well as memory space. Nowadays, however, with the development of high-performance processors and lower-cost memory, these two issues have been alleviated largely. LSI has been successfully implemented in Google and other search engines to provide more related documents in search results for users. Google introduced Latent Semantic Indexing in January 2010.

Choosing an optimal number of dimensions to use when we calculate the SVD in LSI is also challenging. In practice, we usually get broader comparisons of the concepts contained in the document pool using fewer dimensions. On the other hand, we can get more specific comparisons of concepts with more dimensions in calculating SVD. However, the actual number of dimensions available for use is constrained by the number of documents in the pool. [ Source: http://en.wikipedia.org/wiki/Latent_semantic_indexing ]

LSI is not able to capture relevant words in processing a bag of words, i.e., when a text is represented as an unordered collection of words. This is because that  LSI is based on the principle that words tend to be closely related to each other if they appear often close to each other in documents.

Further readings

Following are a list of selected essays and research papers that will help me learn more about Latent Semantic Indexing. Most of the essays are written in plain English and easy to understand. The research papers are well known papers published on this topic in recent years.

Patterns in Unstructured Data
Latent Semantic Indexing at Tennessee Uni
Constructing and Examining Personalized Cooccurrence-based Thesauri on Web Pages
Telcordia Latent Semantic Indexing
Using Latent Semantic Indexing for Information Filtering