Stern Center for Research Computing http://vleda.stern.nyu.edu/scrc New York University • Leonard Stern School of Business Mon, 24 Sep 2012 18:34:37 +0000 en hourly 1 http://wordpress.org/?v=3.0.1 Hadoop 1.0.3 installed, including hive 0.7 and the latest mahout and hbase http://vleda.stern.nyu.edu/scrc/?p=3178 http://vleda.stern.nyu.edu/scrc/?p=3178#comments Mon, 24 Sep 2012 18:34:37 +0000 nwhite http://vleda.stern.nyu.edu/scrc/?p=3178 The Stern Center for Research Computing  has recently upgraded  hadoop and all of it’s related tools to the latest releases.

These include hadoop, hive, pig, hbase, mahout and pegasus.

These tools form the basis for the Stern “big data” infrastructure, which can be used to handle very large  or complex data sets that are not amenable to more traditional data storage and analysis techniques.  There are several pilot projects already underway and we are looking for more.  Soon we expect to have  a Sas-Hive link, so that sas users can store their data in hive and use HQL to manipulate it and then bring it into sas for processing.

See a member of the research computing staff for more information.

]]> http://vleda.stern.nyu.edu/scrc/?feed=rss2&p=3178 0 Links for SAS users http://vleda.stern.nyu.edu/scrc/?p=2841 http://vleda.stern.nyu.edu/scrc/?p=2841#comments Mon, 05 Dec 2011 17:16:47 +0000 nwhite http://leda.stern.nyu.edu/scrc/?p=2841 UCLA SAS materials

https://www.ats.ucla.edu/stat/SAS/sk/default.htm

Is a complete intro and overview to SAS. It has examples, videos, data sets etc.

A good way to both get started with SAS, as well as find advanced techniques.

]]> http://vleda.stern.nyu.edu/scrc/?feed=rss2&p=2841 0 Running hadoop, hive and mahout at the Stern Center for Research Computing http://vleda.stern.nyu.edu/scrc/?p=2278 http://vleda.stern.nyu.edu/scrc/?p=2278#comments Sun, 19 Jun 2011 16:57:21 +0000 admin http://leda.stern.nyu.edu/scrc/?p=2278 First, you must have your Stern userid enabled for hadoop, and a hadoop user  created for your userid. To do that, please send an email to research@stern.nyu.edu, or call the help desk at 212-998-0180 and create a ticket for research computing.

You should be able to run hadoop map-reduce jobs, hive and mahout.

To start, type hadoop fs -lsr  /

and you should get a list of all the files in the file system.

hive

will enter the hive command line environment

mahout will give you access to the mahout command line.

Good luck….

]]> http://vleda.stern.nyu.edu/scrc/?feed=rss2&p=2278 0 Hadoop at Stern http://vleda.stern.nyu.edu/scrc/?p=2274 http://vleda.stern.nyu.edu/scrc/?p=2274#comments Sun, 19 Jun 2011 16:49:11 +0000 admin http://leda.stern.nyu.edu/scrc/?p=2274 Research Computing is now piloting a small hadoop cluster. If successful, we will try to get funds to enlarge it.

Currently there are 6 processing nodes with about 16 cores and about 1TB of disk.

In addition to hadoop and map-reduce, hive and mahout (0.5) are also installed.

To use hadoop, please look at these instructions.

]]> http://vleda.stern.nyu.edu/scrc/?feed=rss2&p=2274 0