What now?

Now that you have a running stack and created a dummy job, you can start thinking of a real life application.

Here is an implementation example of Cloud Transcode in a real world environment.

The app

You have a website that allows users to upload videos to it. Like YouTube. Upon receiving a new video you need to probe it, create thumbnails and alternative video formats.

Initiate a job

Upon receiving a video, your webserver will send a ‘start_job’ message through SQS to the stack which is running somewhere in the Cloud or locally.

If you use PHP you can use the CPE-Client-SDK to easily send this message out. See: https://github.com/sportarchive/CloudProcessingEngine-Client-SDK-PHP

You must keep track of the job you just started now. So your webserver will store the jobID returned by ‘start_job’ (or created by you). Store this jobID in your DB and associated it with your user’s video.

Capture progress

Now you need to know what’s going on in the Stack so you can show statuses back on your site, or initiate something else.

To do this you need a daemon running along side your webserver. This daemon is implemented by you and will constantly poll the SQS output queue for incoming message.

See the client_example folder in the CPE project. There is a ClientPoller.php that does just that.

After the webserver submit a new job, the first message your deamon will receive is a “WORKFLOW_SCHEDULED” message sent by the InputPoller (It receives your ‘start_job’ command and started a SWF workflow for you). This message contains your jobId and also a runId which is the Workflow ID in SWF. You must now track this runId in your DB.

All sub-sequent messages will only contain the runId so it’s imperative that you track it.

You will then start receiving messages from the stack:

  • ACTIVITY_STARTED
  • ACTIVITY_PROGRESS
  • ACTIVITY_FAILED
  • ACTIVITY_COMPLETED
  • etc

You can parse those messages and use the data they contain to update your database and show status and progress on your website. The output of those messages is not yet documented. It’s quite straight forward though so print them out and analyse them.

Advantages

CPE/CT is completly decoupled from your application. There is no performance impact on your webserver and you can scale you transcoding platform at will.

Using CPE/CT requires good knowlege, in AWS SWF and SQS and in development in general.


Next: Activities