Using Link:

If you are like me, you are dealing with a lot of different technologies in your environment. Some are accessed by api, some are database connections, others need special configuration and logic to use. Link lets you “wrap” each piece and centralize the configuration of these pieces. Then, all of these pieces can be accessed in a standardized way, with a tight integration with ipython

Wrappers

You probably noticed the fact that every resource has a wrapper. The wrapper tells link what type of object this resource is. When link creates these resources it will create the object and pass it the parameters you have configured. The wrappers that ship with link are:

APIRequestWrapper
MysqlDB
SqliteDB
HbaseDB
MongoDB

Note, these have dependancy package requirements for them to work

The lnk Singleton

When you get started you want to import lnk, which is a singleton that contains all of the configuration details from ~/.link/link.config:

In [4]: from link import lnk

NOTE:: If your config file is not proper json you will get an error when trying to import lnk

You can look at what is in your configuration using the config() function, which will return a dictionary.

In [3]: lnk.config().keys() Out[3]: [‘dbs’, ‘apis’]

Accessing Configured Resources

The lnk object let’s you treat everything in your configuration as an object. For instance, let’s say we have this as our configuration:

{
    "apis": {
       "my_api": {
           "wrapper": "APIRequestWrapper",
           "base_url": "http://123fakestreet.net",
           "user": "<user>",
           "password": "<password>"
       }
}

I can access my_api by calling lnk.apis.my_api. This may seem strange, but lnk under the hood will cascade through the configuration and create objects:

In [3]: from link import lnk

# Save my_api to the api variable to avoid creating many copies
In [4]: api = lnk.apis.my_api

In [5]: api
Out[5]: <apiwrappers.APIRequestWrapper at 0x10526f390>

Note: every time you do this it makes a NEW APIRequestWrapper, so set it to a variable. You will see below in the iPython integration why this is so powerful:

API Responses to Json and XML:

The APIResponseWrapper has convience functions for json and xml responses:

In [3]: from link import lnk

# Save my_api to the api variable to avoid creating many copies
In [4]: api = lnk.apis.my_api

In [9]: resp = api.get('/api_service?param=blah')

# look at the raw content
In [10]: resp
Out[10]: '{"total":0,"rank":"0","success":true}'

# json deserialize into a dictionary using the json property
In [11]: resp.json['success']
Out[11]: true

In [12]: resp.json['total']
Out[12]: 0

In [43]: type(resp.json)
Out[43]: dict

For xml there is an xml property. It will return the results as pythons xml.etree.cElementTree.

DBConnections

Database connections work the same way:

In [3]: from link import lnk

In [35]: my_db = lnk.dbs.my_db

In [36]: data = my_db.select('select id from my_table')

#returns a cursor wrapper which has some conviennce funtions
In [10]: data
Out[10]: <link.wrappers.dbwrappers.DBCursorWrapper at 0x10b318a50>

    In [12]: [x for x in data]
    Out[12]:
    [(6L,),
    (4L,),
    (9L,),
    (8L,),
    (7L,),
    (3L,),
    (2L,),
    (1L,),
    (12L,),
    (13L,),
    (5L,),
    (10L,),
    (11L,),
    (14L,)]

Queries to Pandas Dataframes

If you don’t know about pandas you are missing out (make sure its installed). You can select any query into Pandas DataFrames using the select function instead of the select function of a DBConnectionWrapper:

In [35]: my_db = lnk.dbs.my_db

In [36]: df = my_db.select('select * from my_table').as_dataframe()

<class 'pandas.core.frame.DataFrame'>
Int64Index: 325 entries, 0 to 324
Data columns:
id               325  non-null values
user_id          323  non-null values
app_id           325  non-null values
name             325  non-null values
body             325  non-null values
created          324  non-null values
dtypes: float64(2), int64(3), object(4)

pandas allows you to do groupbys, sums, aggregations, joins...and much more in memory. For more information see the pandas homepage (TODO put link in here)