Google Summer of Code: Loklak

Abstract

Currently, Loklak.net has it’s own search and social wall capabilities.

alt text
alt text

Loklak.org also has a few front-end apps:

However, they are currently not as feature filled as Loklak.net, so the project aims to add additional visualisations and display options, including a live-streaming tweet wall, and timeline for loklak.org

The project

A front-end app that uses loklak.org for API calls to obtain tweets in json format. Users can deploy their own loklak server or locally or online (checkout the loklak_server github repo for one-click deploy buttons!)
Deploy
Deploy on Scalingo
Deploy to Bluemix

I have previously developed a prototype with meteor.js (github repo) which you can deploy locally or online too!

But upon discussion with the loklak team we concluded it was hard to integrate with loklak.org like the current front-end apps, so I decided to make the front-end app with a front-end framework like Angular or React.

This is a sample of part of the search.json API response and fields we could use for visualisations:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
{
... ,
"search_metadata": {
"itemsPerPage": "100",
"count": "18",
"count_twitter_all": 0,
"count_twitter_new": 18,
"count_backend": 0,
"count_cache": 17,
"hits": 18,
"period": 54379295,
"query": "from:leonmakk",
"client": "118.201.140.222",
"time": 1728,
"servicereduction": "false",
"scraperInfo": "http://kaskelix.de:9000,local"
},
"statuses": [
{
"timestamp": "2016-05-17T05:03:07.163Z",
"created_at": "2016-05-17T04:59:47.000Z",
"screen_name": "leonmakk",
"text": "http://en.wikipedia.org/wiki/PC_Tools_(Central_Point_Software)",
"link": "https://twitter.com/leonmakk/status/732435424145227776",
"id_str": "732435424145227776",
"source_type": "TWITTER",
"provider_type": "SCRAPED",
"retweet_count": 0,
"favourites_count": 0,
"images": [],
"images_count": 0,
"audio": [],
"audio_count": 0,
"videos": [],
"videos_count": 0,
"place_name": "Central Point",
"place_id": "",
"place_context": "FROM",
"place_country": "United States",
"place_country_code": "US",
"place_country_center": [
-83.27110293007973,
35.6452903550329
],
"location_point": [
-122.91642774963586,
42.37596116139363
],
"location_radius": 0,
"location_mark": [
-122.91596190504673,
42.37590151256903
],
"location_source": "PLACE",
"hosts": ["en.wikipedia.org"],
"hosts_count": 1,
"links": ["http://en.wikipedia.org/wiki/PC_Tools_(Central_Point_Software)"],
"links_count": 1,
"mentions": [],
"mentions_count": 0,
"hashtags": [],
"hashtags_count": 0,
"without_l_len": 1,
"without_lu_len": 1,
"without_luh_len": 1,
"user": {
"appearance_first": "2016-05-15T06:28:23.263Z",
"profile_image_url_https": "https://abs.twimg.com/sticky/default_profile_images/default_profile_3_bigger.png",
"screen_name": "leonmakk",
"user_id": "728557565676642306",
"name": "leon",
"appearance_latest": "2016-05-15T06:28:23.263Z"
}
},
{
"timestamp": "2016-05-15T06:28:23.263Z",
},
...
]
}

The project aims to improve on the visualisations by using data from the search API.
Analysis of tweets will be done using javascript libraries like D3.js, to produce interactive visualisations.

The “timestamp”, “retweet_count”, “hashtags” and “favourites_count” data fields will be used to update the app without needing to refresh the page by polling the loklak server and front-end updates in real time. For example we can show:
Top hashtag count & users(by retweet/likes) for each day/ hour/ since the 1st day, using bubble charts:

stacked bar charts to show popularity of selected hashtags, divided into 2 groups for each day

radar charts for selected hashtag over time of day

The “mentions” data field will be used for a directed social graph:

Benefits for loklak

  • More compelling visualizations for loklak which let users make use of loklak_server for their events.
  • An app base for loklak users to easily use data from loklak servers, giving users alternatives to loklak.net when loklak.org is not available, and the simple front end apps on loklak.org, which do not currently update in real time, like loklak.net.
  • An app base in Javascript which encourage developers to contribute to loklak front ends, or modify it for their own needs

Project Plan

Time Task
2 weeks Develop the visualizations, using the existing data from loklak server, and libraries like Highcharts or D3 to display the data.
1 week Develop from ‘outside in’ : Make the shell of the app, ie. the layout and structure
1 week Develop the home page & server data processing to better diff the tweets and reduce the current memory load, which was a challenge for the prototype, and implement lazy loading when the tweets overflow.
1 week Develop the tweet grid wall
1 week Develop the data visualisation combi 1
1 week Develop the data visualisation combi 2
1 week Develop settings page and function for user to add settings
1 week Develop settings page to allow user to add panels
2 weeks Tie up loose ends and make screencasts and documentation

Mockups