近期为了研究ConcurrentHashmap源码,需要对sun.misc.Unsafe这个类做一下全面的研究。
1. sun.misc.Unsafe主要API
http://www.docjar.com/docs/api/sun/misc/Unsafe.html 这个网站里面列举了详细的说明:
Class sun.misc.Unsafe consists of 105
methods. There are, actually, few groups of important methods for manipulating with various entities. Here is some of them:
-
Info. Just returns some low-level memory information.
-
addressSize
-
pageSize
-
-
Objects. Provides methods for object and its fields manipulation.
-
allocateInstance
-
objectFieldOffset
-
-
Classes. Provides methods for classes and static fields manipulation.
-
staticFieldOffset
-
defineClass
-
defineAnonymousClass
-
ensureClassInitialized
-
-
Arrays. Arrays manipulation.
-
arrayBaseOffset
-
arrayIndexScale
-
-
Synchronization. Low level primitives for synchronization.
-
monitorEnter
-
tryMonitorEnter
-
monitorExit
-
compareAndSwapInt
-
putOrderedInt
-
/** * Atomically update Java variable to <tt>x</tt> if it is currently * holding <tt>expected</tt>. * @return <tt>true</tt> if successful */ public final native boolean compareAndSwapInt(Object o, long offset, int expected, int x);
二、支撑JNI调用的底层C++代码
#include "sysdep/locks.h" // Use a spinlock for multi-word accesses class spinlock { static volatile obj_addr_t lock; public: spinlock () { while (! compare_and_swap (&lock, 0, 1)) _Jv_ThreadYield (); } ~spinlock () { release_set (&lock, 0); } };
jboolean
sun::misc::Unsafe::compareAndSwapInt (jobject obj, jlong offset,
jint expect, jint update)
{
jint *addr = (jint *)((char *)obj + offset);
return compareAndSwap (addr, expect, update);
}
compareAndSwap (volatile jint *addr, jint old, jint new_val)
{
jboolean result = false;
spinlock lock;
if ((result = (*addr == old)))
*addr = new_val;
return result;
}
Copyright (C) 2002 Free Software Foundation
This file is part of libgcj.
This software is copyrighted work licensed under the terms of the
Libgcj License. Please consult the file "LIBGCJ_LICENSE" for
details. */
#ifndef __SYSDEP_LOCKS_H__
#define __SYSDEP_LOCKS_H__
typedef size_t obj_addr_t; /* Integer type big enough for object */
/* address. */
// Atomically replace *addr by new_val if it was initially equal to old.
// Return true if the comparison succeeded.
// Assumed to have acquire semantics, i.e. later memory operations
// cannot execute before the compare_and_swap finishes.
inline static bool
compare_and_swap(volatile obj_addr_t *addr,
obj_addr_t old,
obj_addr_t new_val)
{
char result;
#ifdef __x86_64__
__asm__ __volatile__("lock; cmpxchgq %2, %0; setz %1"
: "=m"(*(addr)), "=q"(result)
: "r" (new_val), "a"(old), "m"(*addr)
: "memory");
#else
__asm__ __volatile__("lock; cmpxchgl %2, %0; setz %1"
: "=m"(*addr), "=q"(result)
: "r" (new_val), "a"(old), "m"(*addr)
: "memory");
#endif
return (bool) result;
}
// Set *addr to new_val with release semantics, i.e. making sure
// that prior loads and stores complete before this
// assignment.
// On X86/x86-64, the hardware shouldn't reorder reads and writes,
// so we just have to convince gcc not to do it either.
inline static void
release_set(volatile obj_addr_t *addr, obj_addr_t new_val)
{
__asm__ __volatile__(" " : : : "memory");
*(addr) = new_val;
}
// Compare_and_swap with release semantics instead of acquire semantics.
// On many architecture, the operation makes both guarantees, so the
// implementation can be the same.
inline static bool
compare_and_swap_release(volatile obj_addr_t *addr,
obj_addr_t old,
obj_addr_t new_val)
{
return compare_and_swap(addr, old, new_val);
}
// Ensure that subsequent instructions do not execute on stale
// data that was loaded from memory before the barrier.
// On X86/x86-64, the hardware ensures that reads are properly ordered.
inline static void
read_barrier()
{
}
// Ensure that prior stores to memory are completed with respect to other
// processors.
inline static void
write_barrier()
{
/* x86-64/X86 does not reorder writes. We just need to ensure that
gcc also doesn't. */
__asm__ __volatile__(" " : : : "memory");
}
#endif
-
Memory. Direct memory access methods.
-
allocateMemory
-
copyMemory
-
freeMemory
-
getAddress
-
getInt
-
putInt
-
一、数组分配的上限
Java里数组的大小是受限制的,因为它使用的是int类型作为数组下标。这意味着你无法申请超过Integer.MAX_VALUE(2^31-1)大小的数组。这并不是说你申请内存的上限就是2G。你可以申请一个大一点的类型的数组。比如:
final long[] ar = new long[ Integer.MAX_VALUE ];
这个会分配16G -8字节,如果你设置的-Xmx参数足够大的话(通常你的堆至少得保留50%以上的空间,也就是说分配16G的内存,你得设置成-Xmx24G。这只是一般的规则,具体分配多大要看实际情况)。
不幸的是,在Java里,由于数组元素的类型的限制,你操作起内存来会比较麻烦。在操作数组方面,ByteBuffer应该是最有用的一个类了,它提供了读写不同的Java类型的方法。它的缺点是,目标数组类型必须是byte[],也就是说你分配的内存缓存最大只能是2G。
二、把所有数组都当作byte数组来进行操作
假设现在2G内存对我们来说远远不够,如果是16G的话还算可以。我们已经分配了一个long[],不过我们希望把它当作byte数组来进行操作。在Java里我们得求助下C程序员的好帮手了——sun.misc.Unsafe。这个类有两组方法:getN(object, offset),这个方法是要从object偏移量为offset的位置获取一个指定类型的值并返回它,N在这里就是代表着那个要返回值的类型,而putN(Object,offset,value)方法就是要把一个值写到Object的offset的那个位置。
不幸的是,这些方法只能获取或者设置某个类型的值。如果你从数组里拷贝数据,你还需要unsafe的另一个方法,copyMemory(srcObject, srcOffset, destObject,destOffet,count)。这和System.arraycopy的工作方式类似,不过它拷贝的是字节而不是数组元素。
想通过sun.misc.Unsafe来访问数组的数据,你需要两个东西:
1.数组对象里数据的偏移量
2.拷贝的元素在数组数据里的偏移量
Arrays和Java别的对象一样,都有一个对象头,它是存储在实际的数据前面的。这个头的长度可以通过unsafe.arrayBaseOffset(T[].class)方法来获取到,这里T是数组元素的类型。数组元素的大小可以通过unsafe.arrayIndexScale(T[].class) 方法获取到。这也就是说要访问类型为T的第N个元素的话,你的偏移量offset应该是arrayOffset+N*arrayScale。
我们来写个简单的例子吧。我们分配一个long数组,然后更新它里面的几个字节。我们把最后一个元素更新成-1(16进制的话是0xFFFF FFFF FFFF FFFF),然再逐个清除这个元素的所有字节。
final long[] ar = new long[ 1000 ];
final int index = ar.length - 1;
ar[ index ] = -1; //FFFF FFFF FFFF FFFF
System.out.println( "Before change = " + Long.toHexString( ar[ index ] ));
for ( long i = 0; i < 8; ++i )
{
unsafe.putByte( ar, longArrayOffset + 8L * index + i, (byte) 0);
System.out.println( "After change: i = " + i + ", val = " + Long.toHexString( ar[ index ] ));
}
想运行上面 这个例子的话,得在你的测试类里加上下面的静态代码块:
private static final Unsafe unsafe;
static
{
try
{
Field field = Unsafe.class.getDeclaredField("theUnsafe");
field.setAccessible(true);
unsafe = (Unsafe)field.get(null);
}
catch (Exception e)
{
throw new RuntimeException(e);
}
}
private static final long longArrayOffset = unsafe.arrayBaseOffset(long[].class);
输出的结果是:
Before change = ffffffffffffffff
After change: i = 0, val = ffffffffffffff00
After change: i = 1, val = ffffffffffff0000
After change: i = 2, val = ffffffffff000000
After change: i = 3, val = ffffffff00000000
After change: i = 4, val = ffffff0000000000
After change: i = 5, val = ffff000000000000
After change: i = 6, val = ff00000000000000
After change: i = 7, val = 0
三、sun.misc.Unsafe的内存分配
上面也说过了,在纯Java里我们的能分配的内存大小是有限的。这个限制在Java的最初版本里就已经定下来了,那个时候人们都不敢相像分配好几个G的内存是什么情况。不过现在已经是大数据的时代了,我们需要更多的内存。在Java里,想获取更多的内存有两个方法:
1.分配许多小块的内存,然后逻辑上把它们当作一块连续的大内存来使用。
2.使用sun.misc.Unsafe.allcateMemory(long)来进行内存分配。
第一个方法只是从算法的角度来看比较有意思一点,所以我们还是来看下第二个方法。
sun.misc.Unsafe提供了一组方法来进行内存的分配,重新分配,以及释放。它们和C的malloc/free方法很像:
1.long Unsafe.allocateMemory(long size)——分配一块内存空间。这块内存可能会包含垃圾数据(没有自动清零)。如果分配失败的话会抛一个java.lang.OutOfMemoryError的异常。它会返回一个非零的内存地址(看下面的描述)。
2.Unsafe.reallocateMemory(long address, long size)——重新分配一块内存,把数据从旧的内存缓冲区(address指向的地方)中拷贝到的新分配的内存块中。如果地址等于0,这个方法和allocateMemory的效果是一样的。它返回的是新的内存缓冲区的地址。
3.Unsafe.freeMemory(long address)——释放一个由前面那两方法生成的内存缓冲区。如果address为0什么也不干 。
这些方法分配的内存应该在一个被称为单寄存器地址的模式下使用:Unsafe提供了一组只接受一个地址参数的方法(不像双寄存器模式,它们需要一个Object还有一个偏移量offset)。通过这种方式分配的内存可以比你在-Xmx的Java参数里配置的还要大。
注意:Unsafe分配出来的内存是无法进行垃圾回收的。你得把它当成一种正常的资源,自己去进行管理。
下面是使用Unsafe.allocateMemory分配内存的一个例子,同时它还检查了整个内存缓冲区是不是可读写的:
final int size = Integer.MAX_VALUE / 2;
final long addr = unsafe.allocateMemory( size );
try
{
System.out.println( "Unsafe address = " + addr );
for ( int i = 0; i < size; ++i )
{
unsafe.putByte( addr + i, (byte) 123);
if ( unsafe.getByte( addr + i ) != 123 )
System.out.println( "Failed at offset = " + i );
}
}
finally
{
unsafe.freeMemory( addr );
}
正如你所看见的,使用sun.misc.Unsafe你可以写出非常通用的内存访问的代码:不管是Java里分配的何种内存,你都可以随意读写任意类型的数据。
2. ConcurrentHashMap中使用sun.misc.Unsafe的主要功能
2.1 使用
//方法一:
Constructor<Unsafe> unsafeConstructor = Unsafe.class.getDeclaredConstructor();
unsafeConstructor.setAccessible(true);
Unsafe unsafe = unsafeConstructor.newInstance();
//方法二:
Field f = Unsafe.class.getDeclaredField("theUnsafe");
f.setAccessible(true);
Unsafe unsafe = (Unsafe)f.get(null);
//方法三:
UNSAFE = sun.misc.Unsafe.getUnsafe(); (这个会引发SecurityException)
2.2 关于如何确定一个Java类实例大小的文章
http://article.yeeyan.org/view/104091/62930
2.3 方法 getObjectVolatile
2.4 方法 putOrderedObject
2.5 方法 compareAndSwapObject
2.6 方法 compareAndSwapInt
3. sun.misc.Unsafe源码分析
3.1 源代码的获取
Unsafe类提供了硬件级别的原子操作,Java无法直接访问到操作系统底层(如系统硬件等),为此Java使用native方法来扩展Java程序的功能。具体实现使用c++,详见文件sun.misc.natUnsafe.cc();
sun.misc包的源代码可以在openJDK的源码中找到,在线阅读代码地址如下:
http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/9b8c96f96a0f/src/share/classes/sun/misc/Unsafe.java
这个类里面很多方面都是通过JNI调用底层的操作系统提供的方法,
具体在sun.misc.natUnsafe.cc,你可以访问下面的github,它托管了sun.misc.natUnsafe.cc的源代码
https://github.com/aeste/gcc/blob/master/libjava/sun/misc/natUnsafe.cc
4.sun.misc.Unsafe未来
拒传闻这个API将会在JDK 1.9公布 ( the functionality of sun.misc.Unsafe
is considered to become part of Java's public API in Java 9.)
5. 参考资料
http://java.dzone.com/articles/understanding-sunmiscunsafe
http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot-unsafe/
http://sourceforge.jp/projects/pf3gnuchains/scm/git/pf3gnuchains3x/blobs/osx_fix/libjava/sysdep/i386/locks.h
http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/9b8c96f96a0f/src/share/classes/sun/misc/Unsafe.java
https://github.com/aeste/gcc/blob/master/libjava/sun/misc/natUnsafe.cc