Translate

Wednesday, January 12, 2011

What is HyperGraphDB?

HyperGraphDB (http://www.hypergraphdb.org/) is a storage framework based on generalized hypergraphs as its underlying data model. Borislav Iordanov is a creator for HyperGraphDB. It originated from an AI project (http://www.opencog.org) and its power is partly in its data model and in its open-architecture framework.

Two unique features of HyperGraphDB:
  1. Higher-order, n-ary relations are unique to HyperGraphDB
  2. Open-architecture: there’s a very strong “frameworky” aspect to HyperGraphDB, it’s not a black box with fixed, restrictive data-model. The storage layout is open and documented. One can plugin customized indexing, customized type handling, customized back-end storage, customized distribution algorithms etc.
A HyperGraphDB database is a generalized graph of entities. The generalization is two-fold:
  1. Links/edges "point to" an arbitrary number of elements instead of just two as in regular graphs.
  2. Links can be pointed to by other links as well.
Feature Summary:
  1. Powerful data modeling and knowledge representation.
  2. Graph-oriented storage.
  3. N-ary, higher order relationships (edges) between graph nodes.
  4. Graph traversals and relational-style queries.
  5. Customizable indexing.
  6. Customizable storage management.
  7. Extensible, dynamic DB schema through custom typing.
  8. Out of the box Java OO database.
  9. Fully transactional and multi-threaded, MVCC/STM.
  10. P2P framework for data distribution.
More I learn about it., more interesting it becomes. Still working on examples. Detailed information is coming up soon.

HyperGraphDB Resources:
  1. Home: http://www.hypergraphdb.org/
  2. Downloads: http://www.hypergraphdb.org/downloads
  3. API Doc: http://www.hypergraphdb.org/docs/javadoc/index.html
  4. Q&A (Faqs): http://www.hypergraphdb.org/learn?project=hypergraphdb&page=QuestionsAndAnswers

Thursday, January 6, 2011

Getting Started with MongoDB

Introduction:
The name MongoDB comes from "humongous". Its most notable key-features are: FLOSS (Free/Libre Open Source Software), horizontally scalable, very fast, schema-less, document-oriented, written in C++ with native drivers to most programming languages out there e.g. C, C++, C# & .NET, ColdFusion, Erlang, Factor, Java, JavaScript, PHP, Python, Ruby, Perl, etc. belongs into the so called NoSQL category of DBMSs. The way it works is so that it manages collections (the equivalent to tables in RDBMSs) of JSON (JavaScript Object Notation) documents (the equivalent to rows in RDBMSs) which are stored in a binary format referred to as BSON (Binary JSON).

Following is a comparison of MySQL, Oracle, PostgreSQL (DBMSs) with MongoDB (NoSQL):

MySQL, PostgreSQL,...
----------------------
Server:Port
- Database(s)
- Table(s)
- Row(s)
- Column(s)

- Index
- Join

MongoDB
--------
Server:Port
- Database(s)
- Collection(s)
- Document(s)
- Field(s)

- Index
- embedding and linking

The concept of server, database and index are very similar but the concepts of table/collection, row/document as well as column/field are quite different.
In RDBMSs a table is a rectangle made of columns and rows. Each row has a fixed number of columns, if we add a new column; we add that column to each and every row.
In MongoDB a collection is more like a really big box and each document is like a little bag of stuff in that box. Each bag contains whatever it needs in a totally flexible manner (read schema-less).
Note that schema-less does not equal type-less i.e. it is just that with MongoDB any document has its own schema, which it may or may not share with other documents.
In practice it is normal to have the same schema for all the documents in a collection. The concept of a column in RDBMSs is closest to what we call a field (key/value pair) in MongoDB — note however what we said above: we can create/read/update/delete individual fields for a particular document in a collection. This is different from creating/reading/updating/deleting a column in RDBMSs, which happens for every row in the entire table.
Indexes are more or less the same for RDBMSs and MongoDB. Joins however do not exist in MongoDB but instead we can embed and/or link documents into/with other documents.
Good resources on GUIs can be found http://www.mongodb.org/display/DOCS/Admin+UIs . Also, assuming a mongod process is running on localhost then we can access some statistics at http://localhost:28017/ and http://localhost:28017/_status.

The software implements a protocol called GridFS that is used to store and retrieve files from the database. This file storage mechanism has been used in plug-ins for NGINX and lighttpd.
MongoDB uses memory-mapped files, limiting data size to 2GB on 32-bit machines (64-bit systems have a much larger data size).

Getting Started:
Download MongoDB from link: http://www.mongodb.org/downloads
Unzip downloaded file and follow below instructions.

By default MongoDB will store data in \data\db, but it won't automatically create that folder, so we do so here:
C:\> mkdir \data
C:\> mkdir \data\db

The important binaries for a first run are:
• mongod.exe - the database server
• mongo.exe - the administrative shell
To run the database, click mongod.exe in Explorer, or run it from a CMD window.
Start the administrative shell, either by double-clicking mongo.exe in Explorer, or from the CMD prompt. By default mongo.exe connects to a mongod server running on localhost and uses the database named test. Run mongo --help to see other options.

By now MongoDB should be running. You can connect to it by using following Java samples. Download Java driver from link: https://github.com/mongodb/mongo-java-driver/downloads

Samples:
m = new Mongo();

// Two other ways to connect MongoDB running on localhost
// Mongo m = new Mongo( "localhost" );
// Mongo m = new Mongo( "localhost" , 27017 );

// Authentication (Optional), name and password are valid for the database then auth will be true
//boolean auth = db.authenticate(myUserName, myPassword);

//At this point, the db object will have a connection to MongoDB server for the specified database.
DB db = m.getDB("mydb");

//Getting A List Of Collections
Set colls = db.getCollectionNames();

for (String s : colls) {
System.out.println(s);
}

//Once you have this collection object, you can now do things like insert data, query for data, etc
DBCollection coll = db.getCollection("testCollection");

/*
* Example to insert a document in following format:
* {
* "name" : "MongoDB",
* "type" : "database",
* "count" : 1, "info" : { x : 203, y : 102 }
* }
*/
BasicDBObject doc = new BasicDBObject();

doc.put("name", "MongoDB");
doc.put("type", "database");
doc.put("count", 1);

BasicDBObject info = new BasicDBObject();

info.put("x", 203);
info.put("y", 102);

doc.put("info", info);

coll.insert(doc);

//findOne() operation to get the first document in the collection
DBObject myDoc = coll.findOne();
System.out.println(myDoc);

Advantages of using MongoDB:
MongoDB combines the functionality of key-value stores - which are fast and highly scalable - and traditional RDBMS systems - which provide rich queries and deep functionality. It is designed for problems that are difficult to be solved by traditional RDBMSs, for example databases spanning many servers.
The database is document-oriented so it manages collections of JSON-like documents. Many applications can, thus, model data in a more natural way, as data can be nested in complex hierarchies and still be query-able and indexable.
Right now (October 2010) we can use MongoDB from at least C, C++, C# & .NET, ColdFusion, Erlang, Factor, Java, JavaScript, PHP, Python, Ruby, Perl. Of course, there might be more languages available in the future.
Scalability, fault tolerance and load balancing. Big words! Sounds scary, sounds hard, sounds expensive. All true, except when armed with MongoDB.
Ad hoc queries
In MongoDB, any field can be queried at any time. MongoDB supports range queries, regular expression searches, and other special types of queries in addition to exactly matching fields. Queries can also include user-defined JavaScript functions (if the function returns true, the document matches).
Queries can return specific fields of documents (instead of the entire document), as well as sorting, skipping, and limiting results.
Querying nested fields
Queries can "reach into" embedded objects and arrays. If the following object is inserted into the users collection:
{
"username" : "bob",
"address" : {
"street" : "123 Main Street",
"city" : "Springfield",
"state" : "NY"
}
}

We can query for this document (and all documents with an address in New York) with:
> db.users.find({"address.state" : "NY"})
Array elements can also be queried:
> db.food.insert({"fruit" : ["peach", "pear", "plum"]})
> db.food.find({"fruit" : "pear"})

When not to use MongoDB:
1. We need strict transactional behavior with any query/write (read ACID) as for example often required with applications/problems in the financial/scientific domain. However, please note that for ordinary use cases the level of ACID provided by MongoDB is by and large sufficient.
2. Our data is very relational. In this case one should just stick to one of the many RDBMSs (Relational Database Management Systems) out there.
3. Related to 2, we want to be able to do joins on the server (but can not do embedded objects/arrays).
4. We need triggers on our tables (called collections in MongoDB parlance) — note: there might be triggers available soon.
5. Related to 4, we rely on triggers (or similar functionality) to do cascading updates or deletes. As for #4, this issue probably goes away once triggers are available.
6. We need the database to enforce referential integrity (MongoDB has no notion of this at all).
7. If we currently (October 2010) need 100% per node durability — there will be major improvements coming with version 1.8 however. Go here for more information.
8. Write ahead log. MongoDB currently (October 2010) does not have one simply because it does not need one — we believe true durability can only be achieved with a cluster setup and never with a single node (this is true for any DBMS used). However, since there is great demand/desire for improved per node durability, the decision has been made to improve per node durability with version 1.8 or sooner.
9. Crystal reports is an example of a type of use that MongoDB is not good at: Dynamic aggregation with ad-hoc queries. These reporting systems (business intelligence) require being able to aggregate and apply mathematical expression to multiple joined sets of data (like across collections). This is something that MongoDB can not handle very well, or at all. Data warehousing, large graph representations (with efficient traversal) and many other types of data, and analysis just do not fit well into the restrictions and choices MongoDB has made, but unlike most of those, reporting is a more generic need that is not well supported.

Useful tips:
1. Use 64-bit version. 32-bit version has a limit on about 2.5GB of data stored. Yeah, it’s probably enough for playing around. But when you start configuring your production (or staging) system, remember to choose 64bit flavor, since you can’t just “fix” that later on, you’ll have to reinstall everything.
2. _id and _ns elements have been added automatically by MongoDB to your document. Remember, MongoDB reserves element names that start with _ for internal use.
3. “Ctrl+C“ is a valid way to shutdown MongoDB.

Issues faced:
1. Exception in initAndListen std::exception: dbpath (/data/db/) does not exist, terminating
Solution: I could start it only with exact dbpath argument on Win - >mongod.exe --dbpath c:\data\db

Resources:
1. MongoDB Road Map: http://jira.mongodb.org/browse/SERVER?report=com.atlassian.jira.plugin.system.project:roadmap-panel
2. Helpful link: http://www.markus-gattol.name/ws/mongodb.html
3. Beginner’s tips: http://senko.net/en/mongodb-gotchas/
4. Tutorial: http://www.mongodb.org/display/DOCS/Tutorial
5. MongoDB Java driver usage tutorial: http://www.mongodb.org/display/DOCS/Java+Tutorial
6. Videos: http://www.10gen.com/video/mongosv2010/crazy
7. WIKI: http://en.wikipedia.org/wiki/MongoDB
8. Downloads:
a. MongoDB downloads: http://www.mongodb.org/downloads
9. Drivers Downloads:
a. https://github.com/mongodb/mongo-java-driver/downloads
b. http://api.mongodb.org/
c. http://www.mongodb.org/display/DOCS/Drivers
10. Books: http://www.10gen.com/books
11. Recently added features: http://blog.mongodb.org/post/2388886125/five-new-replica-set-features-in-1-7-x
12. Java Api Docs: http://api.mongodb.org/java/2.5-pre-/index.html

Wednesday, January 5, 2011

BlackBerry Application Development

BlackBerry is a fun mobile platform to start with and it is based on J2ME. Don't think of mobile app development as something new. Till now you might have deployed web apps on web servers and now you will deploy on black berry simulator. As you are a Java developer you will feel at home with BlackBerry too.

First, decide for which OS or mobile model you are going to development. If not sure then avoid working on latest version of OS (OS 6 is the latest). Reason being whatever is developed in OS 6 will only work with latest compatible phones. Few latest APIs may not work in older phones so make sure you start development using OS 4.7 or 5.0 version.

To start BlackBerry app development you need to do following:

1. Download and install BlackBerry eclipse plug-in and its free. Link: http://us.blackberry.com/developers/javaappdev/javaupdate.jsp (If you have done already, run sample and see if simulator is working properly)

2. To deploy app on real device you need to pay onetime fee of 20$. (This is not required for learning purposes)
The online application form for signing keys is available at link: https://www.blackberry.com/SignedKeys/
An administration fee of $20.00 USD will be charged to a valid credit card to complete the registration process. Typically, keys are sent via email within 48 hours of submitting the necessary forms, but occasionally the process can take up to 10 business days. For support contact BlackBerry support at 1-877-255-2377.
You can find more information and register for the code signing keys at
link: http://na.blackberry.com/eng/developers/javaappdev/codekeys.jsp

You can visit following links for more information related to BlackBerry App development:
1. Link: http://us.blackberry.com/developers/started/bbdevapproach.jsp
2. Link: http://us.blackberry.com/developers/started/mobiledev101.jsp

Upload file using servlet

I was looking for easy way to upload excel file from JSP. I came across very good link which contains working example. I had to make few changes in servlet code but thanks a lot for the information provided at the link. See following link for more information.

Title: Servlet File Upload Example
Link: http://www.jsptube.com/servlet-tutorials/servlet-file-upload-example.html