Google Summer of Code: Loklak


Google Summer of Code: Loklak

GSoC []( is a program that encourages open source. The project Loklak is under the FOSSASIA organisation, and is a distributed scraper. The following is my proposal for GSoC.


Currently, has it’s own search and social wall capabilities. also has a few front-end apps. However, they are currently not as feature filled as, so the project aims to add additional visualisations and display options, including a live-streaming tweet wall, and timeline for

The project

A front-end app that uses for API calls to obtain tweets in json format. Users can deploy their own loklak server or locally or online. I have previously developed a prototype with meteor.js (github repo) which you can deploy locally or online too!

But upon discussion with the loklak team we concluded it was hard to integrate with like the current front-end apps, so I decided to make the front-end app with a front-end framework like Angular or React.

This is a sample of part of the search.json API response and fields we could use for visualisations:

 ... ,  
 "search_metadata": {  
 "itemsPerPage": "100",  
 "count": "18",  
 "count\_twitter\_all": 0,  
 "count\_twitter\_new": 18,  
 "count_backend": 0,  
 "count_cache": 17,  
 "hits": 18,  
 "period": 54379295,  
 "query": "from:leonmakk",  
 "client": "",  
 "time": 1728,  
 "servicereduction": "false",  
 "scraperInfo": ",local"  
 "statuses": \[  
 "timestamp": "2016-05-17T05:03:07.163Z",  
 "created_at": "2016-05-17T04:59:47.000Z",  
 "screen_name": "leonmakk",  
 "text": "\_Tools\_(Central\_Point\_Software)",  
 "link": "",  
 "id_str": "732435424145227776",  
 "source_type": "TWITTER",  
 "provider_type": "SCRAPED",  
 "retweet_count": 0,  
 "favourites_count": 0,  
 "images": \[\],  
 "images_count": 0,  
 "audio": \[\],  
 "audio_count": 0,  
 "videos": \[\],  
 "videos_count": 0,  
 "place_name": "Central Point",  
 "place_id": "",  
 "place_context": "FROM",  
 "place_country": "United States",  
 "place\_country\_code": "US",  
 "place\_country\_center": \[  
 "location_point": \[  
 "location_radius": 0,  
 "location_mark": \[  
 "location_source": "PLACE",  
 "hosts": \[""\],  
 "hosts_count": 1,  
 "links": \["\_Tools\_(Central\_Point\_Software)"\],  
 "links_count": 1,  
 "mentions": \[\],  
 "mentions_count": 0,  
 "hashtags": \[\],  
 "hashtags_count": 0,  
 "without\_l\_len": 1,  
 "without\_lu\_len": 1,  
 "without\_luh\_len": 1,  
 "user": {  
 "appearance_first": "2016-05-15T06:28:23.263Z",  
 "profile\_image\_url\_https": "\_profile\_images/default\_profile\_3\_bigger.png",  
 "screen_name": "leonmakk",  
 "user_id": "728557565676642306",  
 "name": "leon",  
 "appearance_latest": "2016-05-15T06:28:23.263Z"  
 "timestamp": "2016-05-15T06:28:23.263Z",  

Benefits for loklak

Project Plan

Time (weeks) Task
2 Develop the visualizations, using the existing data from loklak server, and libraries like Highcharts or D3 to display the data.
1 Develop from ‘outside in’ : Make the shell of the app, ie. the layout and structure
1 Develop the home page & server data processing to better diff the tweets and reduce the current memory load, which was a challenge for the prototype, and implement lazy loading when the tweets overflow.
1 Develop the tweet grid wall
1 Develop the data visualisation combi 1
1 Develop the data visualisation combi 2
1 Develop settings page and function for user to add settings
1 Develop settings page to allow user to add panels
2 Tie up loose ends and make screencasts and documentation

Share on:   Facebook  Twitter