DIY: Liferay Events Hacks: Part 1

Liferay hold many events throughout the year (10+ this year alone), featuring keynotes, workshops, expert speakers, onsite and offsite events, and multiple venues around the world. The data associated with past, current, and future events includes huge agendas, many speakers, rooms, activities, maps, locations, analytics during the event, and much more.

Finding an effective way to aggregate and display all that data for attendees (on multiple devices) is a challenge! We have websites and mobile apps that are designed for displaying this data, but there are things some folks want and others do not, so we are challenging you, the community: do it yourself and make it better than ours, and show everyone what you made!

We're opening up the data stream for our community, and want to see what kinds of new and interesting things you can do with the data. Think dynamic maps. Floating faces. Wordles. Mind maps. Semantic content analysis magic. Mashups with social media. Interactive websites. Ways to connect the dots that no one else thought of. The data is simple, and your creativity is unending, so show us what you can do! (BTW: Technically, the data has been open, just undocumented!)

What's in it for me?

You'll gain respect, love and admiration from our community. Not enough for you? We will also show everyone your creations at the upcoming symposiums and DevCon this fall and tell everyone you are a rockstar!

How do I participate?

Simple - create something awesome using the data feed (see below for details on the format and meaning of the data), and tweet the result (pics, screenshots, videos, links) using the #DIYLiferay hashtag. Also, feel free to blog, forum-post, and anything else you want to use to get the word out. We want to see what you did!

There is a lot of data - you don't have to use all of it. Just pick the stuff you want to use, and ignore the rest. Even just the speaker headshots could be used for some rotating awesome 3D sphere of heads. Also, I tried to describe everything in excruciating detail below, but may have missed stuff, but the names and content should be pretty straightforward, and you can check out the data feed and compare it to the microsite and mobile app to see how the data is used there, to give you some idea on how to use it for your own purposes.

Rules

  • If you write code, seriously consider licensing it with an OSI-approved open source license. We want to learn from you! And find bugs for you.
  • You must follow applicable laws (e.g. copyrights, trademarks, logo usage, etc). Nothing new here.
  • Since the event data changes constantly, you should include the following statement prominently in your solution: "This presentation of Liferay Event data was created by [Your Name Here]. Please note that all data is subject to change and Liferay's official event data can be found at http://liferay.com/events"

Liferay Events Data Feed Details

All event data are stored internally on liferay.com in Dynamic Data Lists, and can be downloaded as individual JSON documents using Liferay's built-in JSONWS service. We layer on a plugin (which will eventually make its way to the Marketplace) to do some post-processing of the returned documents for easier usage in the mobile app.

What code do I need to write?

You can use any language or framework that can retrieve and parse content (so basically, any language!). See the end of this post for working code that you can quickly try out to see how things work. But the main workflow is:

  1. Get a list of events in JSON
  2. Get data about a specific event in JSON
  3. Do something awesome with the data

General guidelines

  • All dates are represented in format YYYY-MM-DD or HH:MM and are assumed to be in the local event timezone unless otherwise noted.
  • Boolean values are represented by the strings 'true' and 'false' - you will need to convert these to real booleans in your chosen language if desired.
  • Some items are themselves are strings that represent JSON documents and must be parsed as such again, in code. For example, you might find a string like "{\"foo\": \"bar\", \"baz\": 3}" and you would need to parse this to programmatically access it (e.g. using JSON.parse()).
  • Some items are lists of name/value pairs, e.g. a=b,c=d and may contain whitespace (including tab characters!) that should be trimmed. That means you should parse 'a=b,c=d' the same as ' a =b , c = d'. Also, colon-separated lists like 'a:b:c,d:e:f' is the same as ' a: b : c ,d: e :f '. Recommendation: use String.trim(). Enjoy!
  • Every record returned via a JSON endpoint will have a UUID associated with it named uuid. This is sometimes used to relate two items together, e.g. agenda items have a list of speakers represented as a list of uuids, which can be used to get the right speaker record via the uuid (assuming you have already retrieved the list of speakers).
  • I suggest you download and cache all event data for a given event at once, and store them internally (possibly using hashtables for quick lookup based on things like uuid). If you only download half the data, you might not be able to de-reference certain pointers (like speaker UUIDs). That's what the mobile app does (and periodically refreshes data once every 30 minutes while the app is running).

Step 1: Getting the list of events

As we have many different events throughout the year, the first thing to get is a list of events, using the following endpoint (notice the hard-coded ddlRecordSetId value - this is your key to the kingdom and should never change for the event listing):

https://www.liferay.com/skinny-web/api/jsonws/skinny/get-skinny-ddl-records?ddlRecordSetId=36416693

Go ahead, click it! You know you want to.

This is the same endpoint used by the mobile app to show you the event select screen when the app is first used. The returned document is a JSON array with the following structure:

[
  { "dynamicElements": EVENT_1_DATA_JSON_OBJECT },
  { "dynamicElements": EVENT_2_DATA_JSON_OBJECT },
  ...,
  { "dynamicElements": EVENT_N_DATA_JSON_OBJECT }
]

Each event is represented as an object in the above array, named dynamicElements. Each event object (the EVENT_N_DATA_JSON_OBJECT from above) has the following elements:

  • active: 'true' or 'false' (note they are strings, not booleans), inactive events aren't finalized and should be ignored (and are blocked in mobile app)
  • booth_map_name: name of map (see below) that represent sponsor booths
  • end_date: when the "thanks for attending" message should show in the app, format YYYY-MM-DD
  • end_time: when the "thanks for attending" message should show in the app, format HH:MM
  • event_hashtag: official event Twitter hashtag (e.g. #lrdevcon for the upcoming developer conference)
  • event_type_dict: translations for select options for sponsor levels, session types, and agenda filter names. Format: comma-separated name=value pairs. E.g. "diamond=DIAMANT,gold=Gold" means the level with key "diamond" should be displayed as DIAMANT (German translation of Diamond) and key "gold" should be displayed as Gold.
  • event_tz: number of hours ahead or behind GMT (negative numbers indicate ahead, e.g. France is -2 in summer, can be used to correct for phones that are set to a different timezone than the event itself to calculate 'local event time' based on device time.
  • event_url: Web URL to the event microsite
  • eventid: identifier used to namespace the user data for this event
  • inactive_msg: message to show user when trying to access inactive event via mobile app
  • latitude: Decimal degrees of latitude for event location (approximate, used for the 'pick the closest location' button in app)
  • location_label: Name of location (usually a city name, e.g. Paris or Boston)
  • logo: the logo to use to represent the event. This string is an embedded JSON document which can be used to construct the final URL to the logo, format is: {"groupId": gid, "uuid":uuid, "version":version}. You can parse this with JSON.parse(). The final URL can then be constructed: https://www.liferay.com/documents/[groupId]/[uuid]
  • longitude: Decimal degrees of longitude for event location (approximate)
  • menutitle: String which appears at top of mobile app screens
  • metadata_types: [see below for details]
  • news_type: ddlRecordSetId of the list of news to show user, can periodically check this to see if there is breaking news (see news item below)
  • ordered_sponsor_levels: the order and size of the sponsor logos on the Sponsors screen. Format: comma-separated list of [name:logos-per-row:size] triplets. For example diamond:1:large,platinum:2:small means that Diamond-level sponsor logos should be shown at the top, 1 per row, and 'large' size, where Platinum-level sponsors are shown 2 per row, and a smaller size. Size can be large or small. Names of sponsor levels are used in the sponsor list, and the display name can be found in the event_type_dict.
  • randomize_sponsors: 'true' or 'false': whether to randomize order of individual logos within each sponsor level.
  • register_url: the URL for people to register to attend.
  • session_survey_questions: A list of questions to ask when survey feedback is given. The format of this is beyond the scope of this challenge :)
  • start_date: when the event begins, used to order on the 'event select' screen. format YYYY-MM-DD
  • start_time: when the event begins, used to order on the 'event select' screen. format HH:MM
  • survey_questions: A list of questions to ask when event feedback is given. The format of this is beyond the scope of this challenge :)
  • title: name of event to show on the event select screen.
  • track_colors: Custom definition of track bar colors for agenda screen. Format: comma-separated list of hex values (where the first one represents track 1, etc) For example #234233,#66DDAA
  • upload_photosetid: the Flickr.com photoset id used to upload and retrieve pictures from the event [see below for details]
  • uuid: unique identifier for this particular object

As new events come online, they will appear in this list. And past events are here as well, in case you wish to test with past event data to fine-tune your app!

The metadata_types element

Each event has a metadata_types field. This is the key to drilling down and getting event-specific data (like agendas, speakers, etc). The format of this element is a comma-separated list of name:ID pairs. For example agenda:12323423,activities:23423243,... You use these IDs to retrieve data of a particular type for a particular event, for example: https://www.liferay.com/skinny-web/api/jsonws/skinny/get-skinny-ddl-records?ddlRecordSetId=35246557 just as you did for the event list. The elements in the metadata_types list include:

  • agenda: A giant list of all agenda items for the event that appear on the microsite and mobile app
  • activities: A list of onsite and offsite activities (e.g. community meetups, social hours, etc)
  • contacts: A list of initial contacts to populate the 'Contacts' screen for the event
  • galleries: A list of photo galleries to show on the 'Gallery' screen for the event
  • maps: A list of maps to show on the 'Maps' screen for the event
  • rooms: A list of rooms used at the event (used to map agenda items to physical rooms)
  • speakers: A list of all speakers (used to generate pics, biographies, etc)
  • sponsors: A list of the sponsors of the event, for the 'Sponsors' screen

Each type of data is explained below. You will also find elements for beacon_forms, beacon_individual_events, beacon_region_events, and beacon_regions: These are related to our iBeacons feature at events, and will be covered in part 2 of this blog post.

Step 2: Fetch data about an individual event

Once you have fetched the list of events above, you can then use the metadata_types object for an event to access details about the specific event, using the ddlRecordSetId's in the metadata_types field for that event. For example, the agenda for France Symposium is 35246557.  You should fetch all data (agenda, speakers, rooms, etc) for a given event at once, because some elements reference other elements (e.g. an agenda item lists speakers by their uuid, which can be looked up in the speakers data for the event).

Fetching The Agenda

The agenda is the most complicated of all the event data types. Here's the France Symposium agenda. Note that all agenda items are represented here, including breaks, after-parties, registration, etc. Some elements may not have all fields filled out (e.g. there are no speakers for the 'Breakfast' agenda item, and no video URLs either). The format is the same as above, with the following elements:

  • custom_css_class: ignore this, not the droid you are looking for
  • date: Day of the session, format: YYYY-MM-DD
  • display_in_mobile_app: whether to display in mobile app or not (some items are not)
  • display_on_live: whether to display on microsite or not (some unfinished items are not, and you should ignore these)
  • download_label: Slide download URL (only active after event)
  • download_url: Session replay URL (only active after event)
  • enable_notes: Whether to enable 'personal notes' field in app (e.g. "Lunch" is not enabled for this)
  • enable_ratings: Whether to enable the session to be rated (e.g. "Afternoon Break" is not rateable)
  • end_time_hour: When the session ends (it's actually a JSON array with a single element representing the hour of end - don't ask why)
  • end_time_minutes: When the session ends (it's actually a JSON array with one element representing the minutes of end time - don't ask why)
  • room_uuid: The UUID of the room element that in which this session takes place
  • session_summary: The human-readable abstract/summary of the session
  • session_type: A single-element JSON array representing the type ("Technical", "Case Study", "Workshop", etc. used to put cool icons on the display)
  • speakers_uuid: A comma-separated list of speaker UUIDs for the session. You can look up speaker details via the UUID and the speaker list below.
  • select_category: If present, then this is a JSON array representing a list of categories to which this session belongs. E.g. all Mobile-related talks would have mobile as a category, or perhaps mobile, responsive. These are free-form tags that are used on the website and mobile app to allow attendees to only show certain kinds of sessions. The display name of the filters can be found in the event_type_dict dictionary for the event.
  • sponsors_uuid: A comma-separated list of sponsors that are sponsoring the session (e.g. for sponsored after-parties, etc)
  • start_time_hour: When it starts (again, a single-value JSON array)
  • start_time_minutes: When it starts (again, a single-value JSON array)
  • survey_questions: The list of questions to ask when rating the session. Out of scope for this challenge!
  • title: The title of the session
  • uuid: The unique identifier for this session
  • video_url: Direct video replay URL (available after event)

Fetching The Speakers

Here's the France Symposium speaker list. Format is the same, with the following elements:

  • speaker_bio: Human-readable biography of speaker
  • speaker_company: Their company
  • speaker_highlight: true or false. Whether they are highlighted. Highlighted speakers are shown on a "Highlighted speakers" page on the microsite.
  • speaker_image: The URL to the headshot. Again, it's a JSON document that can be used to construct the URL in the same way as other images (for example the logo element in the Event List)
  • speaker_keynote: true or false. true means it's a keynote, and is highlighted on the agenda listing.
  • speaker_name: Name of speaker
  • speaker_social_blog: URL to their blog
  • speaker_social_facebook: URL to their Facebook page
  • speaker_social_liferay: URL to their Liferay profile
  • speaker_social_linkedin: URL to their LinkedIn page
  • speaker_social_twitter: URL to their twitter page
  • speaker_social_youtube: URL to their youtube page
  • speaker_title: Their job title
  • uuid: Unique identifer for this speaker

Fetching The Sponsors

Example from France Symposium. Here are the elements:

  • docmedia: The sponsor's logo. Same deal as above, it's a JSON document which can be parsed to construct the URL.
  • level_rank: Their rank within their level. E.g. a Gold sponsor of rank 2 would be displayed below rank 1 or 0. Ignored if randomize_sponsors is in effect for this event.
  • link: The link to the sponsor's homepage/website.
  • name: The name of the sponsor.
  • rank: Their rank - it's a single-element JSON array indicating the key of their rank. e.g. 'diamond' or 'platinum' or 'exhibitor'.
  • type: A single-element JSON array. 'event' is a traditional sponsor. 'photo' is a photostream sponsor. 'scan' is a badge scanner sponsor.
  • uuid: Unique identifier for this sponsor.

Fetching The Rooms

Rooms are linked to sessions via UUID. Here's the France Symposium room list. Elements are:

  • display_as_track_heading: ignored (it used to mean something, but no longer)
  • map_id: Unused currently (we use the name of the room as a key in the other lists)
  • room_name: Name of room (e.g. "Grote Zaal" or "Grand Ballroom")
  • room_number: Unused currently
  • uuid: A unique identifier for the room

Fetching The Activities

Activities are things like Community Meetups, etc that appear on the "Activities" screen in the mobile app. Here's the France Symposium activities list. Elements:

  • date: The date of the event, format: YYYY-MM-DD
  • description: Human-readable description of event
  • endtime: When it ends. Format: HH:MM
  • hostedby: Company or individual that is hosting activity
  • map_name: The name of the map element that can be used to generate a pointer to the map itself.
  • picture: A JSON object that can be used to construct a pointer to the image for the activity.
  • starttime: When it starts. Format: HH:MM
  • title: The title of the activity
  • uuid: A unique identifier for the activity.

Fetching The Maps

Maps are used to show room layouts and provide maps to relevant physical locations (e.g. hotels, pubs, tennis courts, etc). Here's the France Symposium map listing. Elements includE:

  • address: The physical address of the place. Don't try to parse it, just use it as input to some map service like google. Also, for room maps, there's no address.
  • icon: A small thumbnail representing the location.
  • image: A bigger image representing the location.
  • name: The name of the place (is used in the Activities list to generate a link to this map)
  • phone: If you want to call/text the location, use this phone number.
  • show_map: For items that don't have an actual address (like a room in a venue), this is 'false', and the mobile app won't generate a dynamic google map.
  • uuid: A unique identifier for this map. Is used elsewhere (e.g. in the agenda's room_uuid element) to reference this map.

Fetching The Contacts

In the mobile app, there is a Contacts screen with a default list of contacts, that can be added to at certain events that have QR codes printed on badges. Here's the France Symposium contacts list. Elements:

  • blog: Their blog pointer
  • city: The city in which they do business
  • companyname: Name of company
  • country: Name of country
  • facebook: Their facebook page
  • firstname: Their given name
  • googleplus: Their Google+ page
  • lastname: Their family name
  • linkedin: Their LinkedIn page
  • phone: Their business phone number
  • picture: Their image (again, a JSON object that must be parsed to construct an image URL)
  • readonly: Whether they can be deleted. 'true' or 'false'.
  • state: Their state/region
  • street: Their street address
  • twitter: Their twitter page
  • url: Their company's URL
  • uuid: A unique identifier for this contact
  • youtube: Their YouTube page
  • zip: Their postal code

Fetching The Galleries

In the mobile app, there is a Gallery page with different tabs for different events. Generally, the gallery in position 1 is the gallery representing the current event that is being brosed. Here's the France Symposium gallery list. Elements:

  • photosetid: The flickr.com photoset identifier (see below)
  • position: The position of the tab (1=leftmost, 4=rightmost)
  • rateable: Whether pics in this gallery can be rated (thumbs up). Generally only the current event's pics can be rated.
  • title: The title of the gallery (appears on tabs)
  • uuid: A unique identifier for this gallery entry.

During the event, attendees are encouraged to take pics and upload them with the mobile app, and the pics go to Flickr. You can retrieve a listing of all the pics from Flickr using the photosetid and Flickr's REST web service (you have to have an API Key, and follow their docs closely to do this). This listing you get back from Flickr also includes a direct photo URL for each photo, so you can make pretty pictures dance.

And that's it!

Trying it with Sample Code

Since you're reading this on liferay.com, simply open your Developer JavaScript console in your browser, and copy/paste this code into it and press return. Note there's absolutely no error checking whatsoever.

var eventsUrl = 'https://www.liferay.com/skinny-web/api/jsonws/skinny/get-skinny-ddl-records?ddlRecordSetId=36416693';

function listener() {
    var events = JSON.parse(this.responseText);
    var eventList = '';
    events.forEach(function (el) {
        var event = el.dynamicElements;
        eventList += (event.start_date + ' ' + event.title + ' ' + event.location_label + '\n\n');
    });
    alert(eventList.trim());
}

var req = new XMLHttpRequest();
req.onload = listener;
req.open('get', eventsUrl, true);
req.send();

This will generate an alert popup for our events by retrieving the listing and parsing it out:

Another Example

This one is a bit more complicated, it retrieves all of the events, looks for our North America Symposium entry, then retrieves its agenda and looks for sessions that are categorized as mobile sessions, and shows its details in another annoying popup.
var urlPrefix = 'https://www.liferay.com/skinny-web/api/jsonws/skinny/get-skinny-ddl-records?ddlRecordSetId=';

function handleAgenda() {
  var agenda = JSON.parse(this.responseText);
  agenda.forEach(function(info) {
    var agendaItem = info.dynamicElements;
    if (agendaItem.select_category) {
      var cats = JSON.parse(agendaItem.select_category);
      if (cats.indexOf('mobile') != -1) {
        alert('A a mobile session: ' +
              agendaItem.title + ' on ' +
              agendaItem.date);
      }
    }
  });
}

function handleEvents() {
  var events = JSON.parse(this.responseText);
  // find the NAS Boston event
  events.forEach(function(event) {
    if (event.dynamicElements.eventid === 'lr-nas-2014') {
      // get the metadata_id for the agenda and put it into a dictionary
      // dealing with whitespace via trim()
      var eventDict = {};
      event.dynamicElements.metadata_types
        .split(',')
        .map(function(el) {return el.trim();})
        .forEach(function(nvpair) {
          var parts = nvpair.split(':').map(function(el) { return el.trim();});
          eventDict[parts[0]] = parts[1]
        });
      var agendaId = eventDict.agenda;
      var agendaReq = new XMLHttpRequest();
      agendaReq.onload = handleAgenda;
      agendaReq.open('get', urlPrefix + agendaId, true);
      agendaReq.send();
    }
  });
}

var eventsReq = new XMLHttpRequest();
eventsReq.onload = handleEvents;
eventsReq.open('get', urlPrefix + '36416693', true);
eventsReq.send();

Good luck, and don't forget to tweet your creation with #DIYLiferay or leave it in comments below! Happy hacking!

Image credits: DIY Google Glass from FalconHive

Blogs
Where is the data?The url doesn't work.
@Rubén - do not worry - it will be back in 48 hours. We have been upgrading liferay.com and accidentally undeployed the skinny-web plugin that is responsible for feeding you the data. It will be back on Friday!
I've represented a Speakers Liferay Events chart where you can see the speakers classified by Liferay, the company rank and others, you can compare with any other... It's available on www.liferayevents.appspot.com.

I would like to having access to the tweets by event_hashtag on API Console Twitter to do something more interesting but it doesn't work although i can see on twitter.

Finally, i would recommend to have more enumerated type like speaker_highlight, rank, session_type..... an useful field is select_category but it presents in few events.


Thanks!