Skip to main content

Database Design For Google App Engine - Tutorial

Are you looking for app engine database designing tutorials in the Internet? Well unfortunately there is very few information about it out there. Most of them are generic in nature, praising the stale common features of cloud computing we've heard over and over again - scalability, fast, non-relational stuffs in general etc. I kept searching for hours to find that most of the results were not useful to make a real world applications in GAE, and finally stumbled on some very valuable sources and techniques from several sources and gathered a final concepts you need to know to model data in Google app engine. I'll share it very comprehensively in this tutorial so that you don't need to gather it from different places as I did.

I'm quite experienced in relational database ( Oracle and MySQL) previously. I've been trying to design a product in Google App Engine since they provide free Java hosting. It was then that I got stuck wondering how to design my tables and schema in the non-relational database provided by the app engine. Eventually, this is how I learned it.

Before Designing Database in Google App Engine

The Google App Engine uses 'BigTable' as the database, it's a non-relational database with high scale-ability and storage capacity. So learning about some basic concepts of the non-relational database before hand will be very helpful.
Concepts to learn: 

  • Basic difference between relational and non-relational database
  • What is indexing ? 

Creating Tables (Entities) in Google App Engine

Actually, there are no tables in gae. Everything is stored as Entity. There are three ways you can structure your backend.

  • With the low-level datastore api
  • JPA 
  • JDO

Setting relations in Google App Engine

Although gae uses a non-relational database, you can still structure relations if you use JPA or JDO in you backend. This a widely used technique

Querying the Google App Engine Datastore

This is a very important portion. There are two important things that you need to learn about gae bigtable that will simplify the way you store data for your queries.

  • Multivalued list properties
  • Self- merge

Both these concepts are explained with example in the Brett Saltkin's video at Google I/O. The examples are shown in python. If you are a java developer you can see some excellent examples in the book Google App Engine Java and GWT Application Development by Daniel Guermeur and Amy Unruh. This book provides a live example of playing with the Google Datastore and contains all you would ever need to create a highly scalable web application in Google App Engine. If you don't want GWT just ignore that part, the book is worth the money for the Datastore part alone. 

These are the two videos by Brett Slatkin (Google) that explain the app engine datastore in much detail and help you get started. I watched these two videos and read this book. That's it. You're ready to run fast with java on cloud! 

Udacity Course On Web Development in Google App Engine

To learn about scale-ability there is an exclusive free course in Udacity here just on the topic. It's taught by the founder of and he clearly explains the whole scaling things happening in reddit, one of the sites with the highest number of visitors. He shows the entire course demo implementation in gae (and that was a jackpot for me!). They offer the entire course videos free to download here . I've been toiling hard with app engine before I got these resources. So I thought sharing this might help other who are stepping the foot in waters.

Each post takes two days or more of research and three hours of typing to bring out the high quality you see here. If you admire my effort, consider giving +1 to this post and Subscribe me in Facebook.  You can also Follow Me on Twitter to get all the new stuffs on GAE. 


Popular posts from this blog

Building Autonomous Drone with Raspberry Pi and APM 2.8

I am a total newbie to hardware and was pushing my limits to see how far I can reach on with hardware projects (which sparked my interest lately). I have set out on a very ambitions mission to control a drone from raspberry pi.I began the research for this around 2 months ago and had brought a raspberry pi, drone body kit and apm flight controller. The key difference of this project from common drone projects is that I'm trying to avoid the use of and RC and instead use the raspberry pi to control it. 

Hardware Installation Setup:
I am using APM 2.8 and Mission Planner.I am using RPi 3 to control the APM 2.8 via Telem port of APMI am planning to power the apm via the battery to ESC (Electronic Speed Controllers)

Now, documenting my steps below:
Day 1Watch TutorialTo get started with APM flight controller, I watched this video tutorial [1] which gives a gentle introduction about APM board. 
Setup APM board and Calibrate SensorsI downloaded the APM Mission Planner software for mac and in…

Is MacBook Air Good For Programming / Blogging ?

I'm a passionate java developer who just migrated from a Windows PC netbook ( Dell mini ) to a 13 inch MacBook Air. Before the netbook I owned a Dell inspirion 1501. I'm quite a bit of an avid blogger as well. I purchased Dell mini just as it was launched hoping that it's compact and mobile architecture would solve all of my need as a programmer and a writer. Unfortunately it turned out that it was a worthless device.The rest of the story goes...

Do Not Compare a Netbook With MacBook AirMacBook's astonishing features far exceeds anything that of a normal Netbook.Before
Buying a netbook for programming and blogging was one of the biggest blunders I ever made on choosing a machine. The screen was 11 inch and clumsy icons of the Windows were a disgrace all the time.The tightly arranged keys in the keyboard made typing a pain. It's slow Intel Atom Processor is too sluggish to run even VLC player.

After  The Mac's backlit spacious keyboard layout, 1440x900 resolution d…

ICFAI Sikkim Distance MBA Review From My Experience

After a long research I joined for the ICFAI distance MBA program in 2012. Now I've completed 2 semesters ( as of 2013 ). I wanted to write this review so that people who are looking for a good MBA program can get a hand-on review about the distance MBA offered by the ICFAI Sikkim. I've been through all the cycles of this program and this review might help you make the right choice about the program. This article presents my own ( and unbiased ) view of the program and is in no way associated with the course provider.

Is the MBA ICFAI Sikkim Approved By UGC?  As per the latest AICTE regulations, a distance education program must have the approval of a joint commission of  UGC- University Grant CommissionAICTE- All Indian Council for Technical EducationDEC- Distance Education CouncilICFAI Sikkim doesn't have this approval (don't get disappointed, it's not over yet). Only institutes and colleges affiliated to a University are required to take AICTE approval. So ICFAI be…