Wednesday, 23 November 2011

Running GFMon on a 64bit MAC OSX

Here are the quick setup steps to run GFMon 26 on a 64 bit MAC OSX operating system.

1. Change WS (osgi.ws [windowing system]) in the gfmon command from gtk to carbon
2. Add the -d32 switch
3. Add the -XstartOnFirstThread switch

The command in the gfmon script should look like with the changes in BOLD:

${GF_JAVA:-java} -d32 -XstartOnFirstThread -Xms256m -Xmx512m ${JAVA_ARGS} -Xbootclasspath/a:"$GEMFIRE/lib/gemfire.jar" -jar "$gfmon/plugins/org.eclipse.equinox.launcher_1.0.101.R34x_v20080819.jar" -WS carbon $@

4. Copy the following jar (from Eclipse ganymede) to the gfmon plugins directory:

$GFMON_HOME/plugins/org.eclipse.swt.carbon.macosx_3.4.1.v3449c.jar

 

Monday, 14 November 2011

Accessing GemFire Regions from Jython

My previous blog entry showed how to access GemFire Regions from JRuby. This demo below accesses the same regions , this time using jython. The code has identical java calls with the obvious jython syntax rather then JRuby. Here I just show the jython script and output.

Before we run the script we need to set the classpath correctly to pick up the GemFire JAR's as well as the JAR file required to access the Region.

export CUR_DIR=`pwd`
export CLASSPATH=$CUR_DIR/lib/antlr.jar:$CUR_DIR/lib/gemfire.jar:$CUR_DIR/lib/gemfire-quickstart.jar

jython script - gemfire-caching-proxy-client.jy

from java.util import Date
from java.util import Collection
from java.util import Iterator

from vmware.au.se.demo.domain import AllDBObject 
  
from com.gemstone.gemfire.cache import Region 
from com.gemstone.gemfire.cache.client import ClientCache
from com.gemstone.gemfire.cache.client import ClientCacheFactory

from com.gemstone.gemfire.cache.query import Query  
from com.gemstone.gemfire.cache.query import QueryService
from com.gemstone.gemfire.cache.query import SelectResults

print "*********************************"  
print "GemFire 6.6 Example from JYTHON"  
print "*********************************"  
  
print "Started at " + Date().toString()  
  
try:  
  
 ccf = ClientCacheFactory()
 ccf.set("cache-xml-file", "client.xml")  
 cache = ccf.create() 

 allObjectRegion = cache.getRegion("AllObjectRegion")  
 queryService = cache.getQueryService()  
 
 query = queryService.newQuery("SELECT * FROM /AllObjectRegion where owner = 'SCOTT'")  
 print "\n** All OBJECTS with owner = 'SCOTT'\n"  

 result = query.execute() 
 collection = result.asList()
 iter = collection.iterator()
 i = 0  
 
 while iter.hasNext():
  i = i + 1  
  entry = iter.next()
  print "Entry " , i , " : " , entry
  
except:  
    print "Unexpected error:", sys.exc_info()[0]  
    raise  
      
finally:  
    cache.close()
     
print "Ended at " + Date().toString()   

Output as follows
Pas-Apicellas-MacBook-Pro:quickstart-client papicella$ jython gemfire-caching-proxy-client.jy 
*********************************
GemFire 6.6 Example from JYTHON
*********************************
Started at Mon Nov 14 08:02:06 EST 2011

[info 2011/11/14 08:02:07.033 EST <main> tid=0x1] 

...

  udp-recv-buffer-size="1048576"
  udp-send-buffer-size="65535"
  writable-working-dir=""
  

[info 2011/11/14 08:02:07.045 EST <main> tid=0x1] Running in local mode since mcast-port was 0 and locators was empty.

[info 2011/11/14 08:02:07.113 EST <Thread-3 StatSampler> tid=0x15] Disabling statistic archival.

[info 2011/11/14 08:02:07.380 EST <poolTimer-client-2> tid=0x1a] AutoConnectionSource discovered new locators [/10.117.85.62:41111]

[config 2011/11/14 08:02:07.381 EST <main> tid=0x1] Updating membership port.  Port changed from 0 to 49,335.

[config 2011/11/14 08:02:07.443 EST <main> tid=0x1] Pool client started with multiuser-authentication=false

[info 2011/11/14 08:02:07.444 EST <main> tid=0x1] Overridding MemoryPoolMXBean heap threshold bytes 0 on pool CMS Old Gen with 352,321,536

[info 2011/11/14 08:02:07.447 EST <main> tid=0x1] Overridding MemoryPoolMXBean heap threshold bytes 352,321,536 on pool CMS Old Gen with 352,321,536

[info 2011/11/14 08:02:07.447 EST <Cache Client Updater Thread  on Pas-Apicellas-MacBook-Pro(434)<v2>:38304/49301> tid=0x1c] Cache Client Updater Thread  on Pas-Apicellas-MacBook-Pro(434)<v2>:38304/49301 (10.117.85.62:49311) : ready to process messages.

[config 2011/11/14 08:02:07.527 EST <main> tid=0x1] Cache initialized using "jar:file:/Users/papicella/vmware/scripting/demos/jython/gemfire/quickstart-client/lib/gemfire-quickstart.jar!/client.xml".

** All OBJECTS with owner = 'SCOTT'

Entry  1  :  AllDBObject [owner=SCOTT, objectName=BONUS, objectId=74213, objectType=TABLE, status=VALID]
Entry  2  :  AllDBObject [owner=SCOTT, objectName=SALGRADE, objectId=74214, objectType=TABLE, status=VALID]
Entry  3  :  AllDBObject [owner=SCOTT, objectName=PK_EMP, objectId=74212, objectType=INDEX, status=VALID]
Entry  4  :  AllDBObject [owner=SCOTT, objectName=PK_DEPT, objectId=74210, objectType=INDEX, status=VALID]
Entry  5  :  AllDBObject [owner=SCOTT, objectName=DEPT, objectId=74209, objectType=TABLE, status=VALID]
Entry  6  :  AllDBObject [owner=SCOTT, objectName=EMP, objectId=74211, objectType=TABLE, status=VALID]
Entry  7  :  AllDBObject [owner=SCOTT, objectName=JDBC_BATCH_TABLE, objectId=75753, objectType=TABLE, status=VALID]

[info 2011/11/14 08:02:07.742 EST <main> tid=0x1] GemFireCache[id = 1069736291; isClosing = true; created = Mon Nov 14 08:02:07 EST 2011; server = false; copyOnRead = false; lockLease = 120; lockTimeout = 60]: Now closing.

[info 2011/11/14 08:02:07.771 EST <main> tid=0x1] Resetting original MemoryPoolMXBean heap threshold bytes 0 on pool CMS Old Gen

[config 2011/11/14 08:02:07.803 EST <main> tid=0x1] Destroying connection pool client
Ended at Mon Nov 14 08:02:07 EST 2011  

Wednesday, 9 November 2011

Accessing GemFire Regions from JRuby

Here is a quick demo showing how easy it is to access GemFire regions from JRuby clients. It's assumed you have cache servers started and in this example the cache servers and JRuby client use a locator for connection. It's worth noting the JRuby client is setup as a PROXY client with no client side cache.

Our cache server nodes use a config file as follows - server.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE cache PUBLIC
    "-//GemStone Systems, Inc.//GemFire Declarative Caching 6.6//EN" 
"http://www.gemstone.com/dtd/cache6_6.dtd">
<cache>
  <disk-store name="ds1" auto-compact="true" max-oplog-size="1024" queue-size="10000" time-interval="15">
 <disk-dirs>
  <disk-dir dir-size="4096">persistData</disk-dir>
 </disk-dirs>
  </disk-store>
  <region name="AllObjectRegion"> 
    <region-attributes refid="PARTITION_PERSISTENT" disk-store-name="ds1">
      <partition-attributes redundant-copies="1" />
      <eviction-attributes>
        <lru-heap-percentage action="overflow-to-disk" />
      </eviction-attributes>
    </region-attributes>
    <index name="ownerIdx">
      <functional from-clause="/AllObjectRegion" expression="owner"/>
    </index>
  </region>
  <function-service>
 <function>
   <class-name>vmware.au.se.demo.SizeFunction</class-name>
 </function>
  </function-service> 
  <resource-manager critical-heap-percentage="75" eviction-heap-percentage="65"/>
</cache>

Our JRuby client uses a client config as follows - client.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE client-cache PUBLIC
    "-//GemStone Systems, Inc.//GemFire Declarative Caching 6.6//EN" 
"http://www.gemstone.com/dtd/cache6_6.dtd"> 
<client-cache>
 <pool name="client" subscription-enabled="true">
     <locator host="localhost" port="41111"/>
 </pool>
 <!-- No cache storage in the client region because of the PROXY client region shortcut setting. -->
    <region name="AllObjectRegion">
  <region-attributes refid="PROXY" />
    </region>
    <resource-manager critical-heap-percentage="75" eviction-heap-percentage="65"/>
</client-cache> 

Our JRuby code is as follows - gemfire-caching-proxy-client.rb
require 'java'
require File.dirname(__FILE__) + '/lib/gemfire.jar'
require File.dirname(__FILE__) + '/lib/antlr.jar'
require File.dirname(__FILE__) + '/lib/gemfire-quickstart.jar'

import java.util.Collection;
import java.util.Iterator;

import "vmware.au.se.demo.domain.AllDBObject";

import com.gemstone.gemfire.cache.Region;
import com.gemstone.gemfire.cache.client.ClientCache;
import com.gemstone.gemfire.cache.client.ClientCacheFactory;
import com.gemstone.gemfire.cache.query.Query;
import com.gemstone.gemfire.cache.query.QueryService;
import com.gemstone.gemfire.cache.query.SelectResults;

puts "*********************************************************"
puts "GemFire 6.6 Proxy Client Example from JRUBY"
puts "*********************************************************"

print "Started at ", Time.now, "\n"

begin

 ccf = ClientCacheFactory.new
 ccf.set("cache-xml-file", "client.xml")
 cache = ccf.create
 
 allObjectRegion = cache.getRegion("AllObjectRegion")
 queryService = cache.getQueryService
 
 query = queryService.newQuery("SELECT * FROM /AllObjectRegion where owner = 'SCOTT'")
 print "\n** All OBJECTS with owner = 'SCOTT'\n"
 
 result = query.execute
 collection = result.asList
 iter = collection.iterator
 i = 0
 
 while iter.hasNext
   i = i + 1
   entry = iter.next
   print "Entry ", i , " : " , entry , " \n"
   
 end
 
    cache.close
    
rescue 
  print "\n** Error occured **\n"
  print "Failed to obtian data from gemfire region ", $!, "\n\n"
  
end

print "\nEnded at ", Time.now, "\n"

Output as follows

Note: Some of the gemfire output is omitted to make it a little more readable.

Pas-Apicellas-MacBook-Pro:quickstart-client papicella$ jruby gemfire-caching-proxy-client.rb 
*********************************************************
GemFire 6.6 Proxy Client Example from JRUBY
*********************************************************
Started at Wed Nov 09 12:10:29 +1100 2011

[info 2011/11/09 12:10:29.319 EST <main> tid=0x1] 
  ---------------------------------------------------------------------------
  
....

  udp-recv-buffer-size="1048576"
  udp-send-buffer-size="65535"
  writable-working-dir=""
  

[info 2011/11/09 12:10:29.335 EST <main> tid=0x1] Running in local mode since mcast-port was 0 and locators was empty.

[info 2011/11/09 12:10:29.392 EST <Thread-1 StatSampler> tid=0xf] Disabling statistic archival.

[info 2011/11/09 12:10:29.645 EST <poolTimer-client-2> tid=0x14] AutoConnectionSource discovered new locators [/10.117.85.62:41111]

[config 2011/11/09 12:10:29.646 EST <poolTimer-client-3> tid=0x15] Updating membership port.  Port changed from 0 to 50,522.

[config 2011/11/09 12:10:29.669 EST <main> tid=0x1] Pool client started with multiuser-authentication=false

[info 2011/11/09 12:10:29.671 EST <main> tid=0x1] Overridding MemoryPoolMXBean heap threshold bytes 0 on pool CMS Old Gen with 344,064,000

[info 2011/11/09 12:10:29.673 EST <main> tid=0x1] Overridding MemoryPoolMXBean heap threshold bytes 344,064,000 on pool CMS Old Gen with 344,064,000

[info 2011/11/09 12:10:29.673 EST <Cache Client Updater Thread  on Pas-Apicellas-MacBook-Pro(2183)<v2>:37232/49977> tid=0x16] Cache Client Updater Thread  on Pas-Apicellas-MacBook-Pro(2183)<v2>:37232/49977 (10.117.85.62:49987) : ready to process messages.

[config 2011/11/09 12:10:29.753 EST <main> tid=0x1] Cache initialized using "jar:file:/Users/papicella/vmware/scripting/demos/jruby/gemfire/quickstart-client/./lib/gemfire-quickstart.jar!/client.xml".

** All OBJECTS with owner = 'SCOTT'
Entry 1 : AllDBObject [owner=SCOTT, objectName=EMP, objectId=74211, objectType=TABLE, status=VALID] 
Entry 2 : AllDBObject [owner=SCOTT, objectName=PK_EMP, objectId=74212, objectType=INDEX, status=VALID] 
Entry 3 : AllDBObject [owner=SCOTT, objectName=JDBC_BATCH_TABLE, objectId=75753, objectType=TABLE, status=VALID] 
Entry 4 : AllDBObject [owner=SCOTT, objectName=PK_DEPT, objectId=74210, objectType=INDEX, status=VALID] 
Entry 5 : AllDBObject [owner=SCOTT, objectName=BONUS, objectId=74213, objectType=TABLE, status=VALID] 
Entry 6 : AllDBObject [owner=SCOTT, objectName=SALGRADE, objectId=74214, objectType=TABLE, status=VALID] 
Entry 7 : AllDBObject [owner=SCOTT, objectName=DEPT, objectId=74209, objectType=TABLE, status=VALID] 

[info 2011/11/09 12:10:29.962 EST <main> tid=0x1] GemFireCache[id = 1691463635; isClosing = true; created = Wed Nov 09 12:10:29 EST 2011; server = false; copyOnRead = false; lockLease = 120; lockTimeout = 60]: Now closing.

[info 2011/11/09 12:10:29.986 EST <main> tid=0x1] Resetting original MemoryPoolMXBean heap threshold bytes 0 on pool CMS Old Gen

[config 2011/11/09 12:10:30.015 EST <main> tid=0x1] Destroying connection pool client

Ended at Wed Nov 09 12:10:30 +1100 2011  

Monday, 7 November 2011

Viewing All MBeans for a GemFire Cluster

In order to view all the MBeans available in a GemFire cluster you could write a simple console client application as follows. This will extract all the MBeans by name and hence work out which one you want to take a look at.

1. First we need to start a JMX agent. The JMX Agent can be run as a separate, “invisible” distributed system member, or it can be collocated in an application. The JMX Agent manages only a single distributed system. 

Here we use a located to join the distributed system

> agent start -J-Xmx550m -dir=agent mcast-port=0 locators=localhost[41111]

2. By default the JMX service URL is as follows

service:jmx:rmi://{agent-host}/jndi/rmi://:{agent-port}/jmxconnector

3. Write a Java Client as follows
package pas.au.gemfire.jmx;

import java.io.IOException;
import java.net.MalformedURLException;
import java.util.Set;

import javax.management.MBeanServerConnection;
import javax.management.ObjectName;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;

public class ShowAllMBeans 
{
    /** The JMX client connector */
    private JMXConnector jmxConnector;

    /** The JMX MBean server connection */
    private MBeanServerConnection mbs;
    
    public void run()
    {
     String urlString = "service:jmx:rmi://localhost/jndi/rmi://:1099/jmxconnector";
     JMXServiceURL url = null;
     
     try 
     {
   url = new JMXServiceURL(urlString);
   jmxConnector = JMXConnectorFactory.connect(url);
   mbs = jmxConnector.getMBeanServerConnection();
   
   // show all MBeans here...
   Set<ObjectName> mbeans = mbs.queryNames(null, null);
   System.out.println(mbeans.size() + " MBeans found\n");
      for (ObjectName mbeanName : mbeans) 
      {
          System.out.println("-> " + mbeanName);
      }   
   
  } 
     catch (MalformedURLException e) 
     {
   // TODO Auto-generated catch block
   e.printStackTrace();
  } 
     catch (IOException e) 
     {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }
     finally
     {
      disconnectFromAgent();
     }
     
     System.out.println("all done...");
    }

 private void disconnectFromAgent()
 {
     if(this.jmxConnector != null) 
     {
       try 
       {
   this.jmxConnector.close();
       } 
       catch (IOException e) 
       {
   // TODO Auto-generated catch block
   e.printStackTrace();
       }
     }
 }
 
 /**
  * @param args
  */
 public static void main(String[] args) 
 {
  // TODO Auto-generated method stub
  ShowAllMBeans test = new ShowAllMBeans();
  test.run();
 }

}

4. Output as follows

292 MBeans found

-> GemFire.Statistic:source=10.117.85.62(23883)-32249/50418,type=IndexStats,name=ownerIdx,uid=130
-> GemFire.Statistic:source=10.117.85.62(23883)-32249/50418,type=IndexStats,name=ownerIdx,uid=131
-> GemFire.Statistic:source=10.117.85.62(23883)-32249/50418,type=IndexStats,name=ownerIdx,uid=132
-> GemFire.Statistic:source=10.117.85.62(23883)-32249/50418,type=IndexStats,name=ownerIdx,uid=133
-> GemFire.Statistic:source=10.117.85.62(23883)-32249/50418,type=IndexStats,name=ownerIdx,uid=134
-> GemFire.Statistic:source=10.117.85.62(23883)-32249/50418,type=IndexStats,name=ownerIdx,uid=135
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=IndexStats,name=ownerIdx,uid=115
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=IndexStats,name=ownerIdx,uid=116
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=IndexStats,name=ownerIdx,uid=113
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=IndexStats,name=ownerIdx,uid=114
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=IndexStats,name=ownerIdx,uid=111
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=IndexStats,name=ownerIdx,uid=112
-> GemFire.Statistic:source=10.117.85.62(23883)-32249/50418,type=DLockStats,name=dlockStats,uid=18
-> connectors:protocol=rmi
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=IndexStats,name=ownerIdx,uid=110
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=IndexStats,name=ownerIdx,uid=119
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=IndexStats,name=ownerIdx,uid=117
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=IndexStats,name=ownerIdx,uid=118
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=VMMemoryPoolStats,name=Par Eden Space-Heap memory,uid=7
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=DiskRegionStatistics,name=/AllObjectRegion,uid=20
-> GemFire:type=MemberInfoWithStatsMBean
-> GemFire.Cache:name=default,id=1558631662,owner=10.117.85.62(23876)-41281/50410,type=Cache
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=DLockStats,name=dlockStats,uid=18
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=VMStats,name=vmStats,uid=3
-> GemFire.Statistic:source=10.117.85.62(23876)-41281/50410,type=IndexStats,name=ownerIdx,uid=102

For more information on using JMX to administer GemFire see the link below.

http://www.gemstone.com/docs/html/gemfire/6.0.0/SystemAdministratorsGuide/JMX.9.1.html