Valtira Blog
HandlerSocket with Java
by Morgan Catlin - 10 Mar 2011, 21:15:51
Ok, huge database nerd update here - I've been keeping my eye on HandlerSocket and how it's been moving toward mainstream MySQL development and implementation. If you haven't already heard - it's a NoSQL implementation allowing you blazing fast access to MySQL tables. It is a plugin that you install in MySQL which allows you to skip all the overhead of executing SQL. Yet, you still get all the ACID-compliant goodness of MySQL (with InnoDB tables). Here, I'll discuss how to install MySQL w/HandlerSocket, set it up and then hook your Java applications to it.
Percona recently added HandlerSocket to their default installation package. I highly recommend using Percona over standard MySQL anyways as it has significant performance, scalability and usage improvements. I'm installing Percona's MySQL over a currently running MySQL server. Step one is to simply download the Percona packages (for your Linux distro - I use Ubuntu) and install it. It will replace the current MySQL binaries and leave your databases and tables alone. On Ubuntu:
gpg --keyserver hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A
gpg -a --export CD2EFD2A | apt-key add -
- Add the following (based on your version) to /etc/apt/sources.list
deb http://repo.percona.com/apt hardy main
deb-src http://repo.percona.com/apt hardy main
- Install using apt - follow the instructions
sudo apt-get install percona-server-server-5.1 percona-server-client-5.1
You should now have Percona installed:
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 9891
Server version: 5.1.55-rel12.6-log (Percona Server (GPL), 12.6 , Revision 200)
Copyright © 2000, 2010, Oracle and/or its affiliates. All rights reserved.
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL v2 license
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
Now, configure HandlerSocket. Edit your /etc/mysql/my.cnf file and drop these under the mysqld section:
loose_handlersocket_port = 9998
loose_handlersocket_port_wr = 9999
loose_handlersocket_threads = 16
loose_handlersocket_threads_wr = 1
open_files_limit = 65535
Now restart your MySQL server. HandlerSocket is now running on port 9999. KEEP IN MIND THAT THIS IS COMPLETELY UNSECURED! ANY IMCOMING CONNECTION SKIPS MySQL's AUTHENTICATION.
Here's a quick test to see if it's working:
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
P 0 mysql user PRIMARY Host,User
0 1
0 = 2 localhost root
0 2 localhost root
Now for the fun part, getting your application connected! First, we'll download and install (with maven) hs4j and dasein-persist. After that, you should have two JAR files to include into your build or enter as dependencies into your maven pom.xml.
If you haven't already, install maven, svn and git for your system. I'm on a Mac, so I ran the installs off of Mac Ports. Some day soon, there will be binaries to download, so you'll be able to skip these steps. Now download the source for hs4j and install:
cd hs4j
mvn install -Dmaven.test.skip=true
You may need to disable the GPG signing plugin in the pom.xml if you don't intend to sign the output from maven install.
Next, download dasein-persist and install:
cd dasein-persist
mvn install -Dmaven.test.skip=true
The JAR files will be under the target directories of both packages. Otherwise, drop these into your pom.xml (you actually only need dasein, but if you want to use hs4j directly...):
<groupId>com.googlecode.hs4j</groupId>
<artifactId>hs4j</artifactId>
<version>0.1-SNAPSHOT</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.dasein</groupId>
<artifactId>dasein-persist</artifactId>
<version>2011.04-SNAPSHOT</version>
</dependency>
You'll need to configure your dasein-persistence.properties file with the HandlerSocket connection details:
dasein.persist.handlersocket.database=mysql
dasein.persist.handlersocket.host=localhost
dasein.persist.handlersocket.port=9999
dasein.persist.handlersocket.poolSize=5
You can use the RelationalHSCache to connect to HandlerSocket and make your normal SQL calls at the same time! Fantastic! Makes moving your Dasein Persist/MySQL implementations super easy!
RelationalHSCache users = (RelationalHSCache) PersistentCache.getCache(User, User.USER_ID);
// Now hit the PRIMARY index
Collection<User> results = users.hsFind("PRIMARY", "localhost", "root");
// Now use that Collection
for (User : user) {
System.out.println("Found user: " + user.getUserId() + " " + user.getHost());
}
We'll be making many modifications to the way Dasein handles HandlerSocket, but this will at least get you to be able access your tables directly. Currently, it doesn't do insert, update or delete, but we'll get there!
Comments? Suggestions? Feedback?
Cloud Applications Presentation
by Morgan Catlin - 16 Nov 2011, 21:10:33
I've submitted my MN IT Symposium presentation, "Architecting and Deploying Scalable, Resilient, and Cost-Effective Cloud Applications", which discusses how to bring about an affective Cloud-based application.
It discusses my experiences using PaaS and rolling my own architecture. I go over what worked and some common misconceptions about using PaaS and RYO.
I think the most interesting part is showing an ideal distributed architecture for the common Java Pet Store application. I talk about how to use AWS' services to achieve high reliability and scalability.
Can't wait to meet all you great attendees!
Comments? Suggestions? Feedback?
Make "Fast" MySQL Backups
by Morgan Catlin - 16 Nov 2011, 21:34:07
Just a quick note, it's not only a good idea to do full MySQL backups but make "fast" ones too.
This may not be possible for you but for me, I do store a lot of data that is used for reporting and generates some large indexes. These tables aren't necessary for my application to run and can be added to a recovery database at a later time using the full backups.
This cuts my recovery time to a new database from hours to minutes. My availability numbers skyrocketed (99.999% is great!).
What I did to create a the fast backup was to do a mysqldump and used --ignore-table to exclude all those reporting tables. Then, executed another mysqldump listing those tables and added the --no-data switch. This creates one dump file with all the necessary data I need as well as the empty tracking tables!
mysqldump --no-data -u admin -p$pwd test_db --tables huge_table >> $file
gzip -9 $file
This gives my a nice small and fast recovery file! Also ideal for making development copies of a production environment!
Categories
- 3d (1)
- ad revenue (1)
- apps (6)
- aws (5)
- branding (1)
- cloud (10)
- cloud computing (1)
- commoditized workloads (1)
- css3 (1)
- development (4)
- ec2 (9)
- handlersocket (1)
- html5 (1)
- innocent bystander syndrome (1)
- integration solutions (1)
- ios (1)
- java (5)
- landing pages (1)
- lead scoring (1)
- marketing (2)
- mn it symposium (1)
- mysql (3)
- nosql (1)
- personalization (1)
- roku (3)
- s3 (2)
- saas (4)
- salesforce (6)
- sociable company (1)
- social (1)
- social business transformation (1)
- social media (1)
- tomcat (2)
- tools (1)
- video (3)
- wowza (3)
Archives By Month
- April 2013 (1)
- March 2013 (1)
- February 2013 (1)
- January 2013 (2)
- October 2012 (1)
- May 2012 (2)
- April 2012 (1)
- January 2012 (1)
- November 2011 (4)
- August 2011 (1)
- July 2011 (1)
- March 2011 (2)
- February 2011 (2)
- January 2011 (1)
- December 2010 (1)
- November 2010 (1)
- October 2010 (2)
- September 2010 (1)
About Us
Valtira's team includes many bright people with useful opinions on the latest web technologies like HTML5 and cloud computing.