Mnemonics are cool techniques that are immensely helpful remembering facts. Be it
anything from spelling of english words to quantum physics, you can make a mnemonic to learn it. For example: The spelling of 'SEPARATE' is one of the the most commonly misspelled word in English language.
"There is A RAT in sepARATe" - visualizing and remembering this sentence will help anyone recall the spelling of separate quickly without mistake.
Another example is the spelling of 'necessary'Just remember the sentence 'Shirts have 1 collar and 2 sleeves' to avoid misspelling the number of 'c' and 's'.
It's a clever idea to keep these tricks under your sleeves to impress your friends or teachers with your new exceptional memory powers.
A trick to recall the first10 element of periodic table is to just remember : "Henry Hester Likes Beer But CanNot Obtain Food Now" Elements: Hydrogen, Helium, Lithium, Beryllium, Boron, Carbon, Nitrogen, Oxygen,Fluorine
Every student uses such tricks every now and then, but they cannot spend too much time making such short-codes (known as mnemonics) for all topics ( because they have loads to study already).
If, all of them have a single platform to share all these tricks then they will have greatly benefit from it and all those students will start thinking creatively about studies. We can reduce their burden and bring a smile to their face while studying.
That is why I've built Spellogram.com a free platform for student's to share and discover the mnemonics.
Map and Reduce acitivity and features like data locality.
Can be applied with a variety of algorithms
Huge data processing can beat good algorithms
Chap-2 - MapReduce
The Map Java class and Reducer Java class
The Job java class
Jobtracker and tasktracker
Hadoop reduces the input to input splits or just splits
Map tasks write the intermediate output to local disks, so that they can be discarded after use.
Outputs of Reduce tasks are stored in HDFS
Combiner function can be run on map output, and the combiner functions output forms the input to the reduce function
Hadoop streaming proivide hadoop apis in languages other than Java
Chap-3 - The Hadoop Distributed Filesystem
Fault tolerant solution. Same data written at multiple places.
Filesystems that manage the storage across a network of machines are called distributed filesystems.
Blocks - a block size is the minimum amount of data it can read and write (for hdfs its 64mb by default)
Namenodes and Datanodes - An HDFS cluster has a master-worker pattern: a namenode (master) and number of datanodes(workers). Master has all the meta data and datanode has all the blocks (but not persistent). Its reconstructed at start time.
On large clusters the time it takes for a namenode to start from cold can be upto 30 mins
Fencing and failover - When one node fails an entity called 'failover controller' switch to the standby node. But first a ZooKeeper is used to ensure that only one namenode is active.
Graceful failover - triggered by adming
Ungraceful failover - in this case to make sure that the other node has completely stopped running, a mechanism called fencing is done. In worst case it does ' shoot the other node in the head' - force shutdown .
File Operations in HDFS
There are java endpoints to do all operations like create, delete, sync
Use Flume and Sqoop to move data
Copy parallel with distcp
Hadoop archives are compressed blocks that can be used as input to MapReduce
Chap - 4 I/O
Reading compressed data
Serialzation in natively implemented in Hadoop for better perfomance
Apache Avro is a project to do this in an improved way and support multiple languages, diff from Google Protocol Buffer and Thrift
Chapter - 5 - Developing a MapReduce Application
Setting up the Environment
- The Configuration API to read xml resource files etc
- Writing Unit Test with MRUnit
- Running locally on a small data - Using Tool Interface write a Driver to run our MapReduce Job (Java file)
- Testing the driver
- Run in Cluster - Package jar - Launching a Job run the driver
- Debugging a Job
- Running multiple Job in particular flow
Chapter 6 - How MapReduce Works
Chapter 9 - Chapter 15
Setting Up Hadoop Cluster
- Using a CDH distribution (See Appendix)
Hadoop Tools :
Pig: Aimed to provide data structure and transformation more than just map and reduce can do
Hive: Made to run queries for people who were weak in Java but strong in SQL
Hbase: Distributed, column-oriented database built on top of HDFS. It is built to scale.
ZooKeeper: Is build to avoid partial failures of request transfers happening between nodes.
Squoop: To transfer data from external applicaitons , web api etc. This is focused on data movement.
The Apple Watch launch is almost nearing at the time of writing this article. I'm all excited and ready to submit my first Apple Watch compatible application to the AppStore. I'll write down my learning experience here so that you can publish your own Apple Watch application to the app store. I'll do this step by step, as the work of my current app progress. This article will be updated over time until I reach the final step to see it live in the AppStore. Step 1 : Make the iPhone Part of the Apple Watch An Apple Watch app is not much different from an iPhone app. In fact, it is a sub-part of the main iPhone application running on the iPhone and the Watch App merely acts as the extension of the parent app in the iPhone. So essentially, need an iPhone app anyway. In this scenario, I'm thinking of building an app that will be useful both on iPhone as well as the Apple Watch, instead of solely focusing on the the Apple Watch aspect. The app will be very simple, but all it has will be available on both the devices. Update: To submit Apple Watch Application now, you should use Xcode 6.2 and not Xcode 6.3. The Xcode 6.3 comes with Swift 1.2 which is currently under beta and not yet supported for AppStore release. Also, as of now Apple AppStore is not accepting Apple Watch applications . See the screenshot from Apple WatchKit portal below:
Apple Watch is soon to release, and being a huge Apple Evangelist, I've been very eager to explore the possibilities of what one can make with these 'Most personal device ever made by Apple'. My hopes are high. I'm in a constant mission to excavate this area at the earliest. And guess what, me and my friends have been working on a couple of interesting 'Apple Watch' things lately:
Both are in beta and will be public soon. I'm trying to get the latest releases of the WatchKit sdk (which is bundled in Xcode 6.2 beta and higher and is required to build Watch Apps) and trying to publish tutorials on new APIs as and when they come.
Following are my objectives with both these online ventures
Teach things more by examples and less by theory
Convey ideas at the simplest form possible
Keep user-interaction at the heart of all materials
Make the examples in a way encouraging the user to replicate the same
Provide quick support in the discussion forum of the course
Let me know the feedback/ideas , and yes watch out this space for more details coming soon.
P.S. : If you'd like to get a free coupon for my course, please leave a comment with your email address below.