Preview Socket

From The Foundry MODO SDK wiki
Jump to: navigation, search

Preview Socket is a protocol for extracting real-time images from modos' preview renderer. Preview Socket clients are separate applications that communicate with modo over a TCP/IP network connection or through pipes and implements the Preview Socket protocol.

While the name implies a TCP (socket) connection, a named pipe variant is also supported. For the purposes of this article, "Preview Socket" refers to both sockets and pipes unless otherwise indicated (usually by through the word "TCP" or "pipe').

Note that Preview Socket is only used for obtaining preview renders and setting up said renderer. Changes to the actual scene must be done through commands, either through modo itself or through Telnet.

Server Commands

High-level control of the Preview Socket server is done through commands.

  • PreviewSocket.listen is used to start or stop listening for TCP connections on the given port. In other words, this starts and stops the Preview Socket server. Any number of clients can connect to the server, with each getting their own preview renderer.
  • Preview.suspend pauses all preview instances, including sockets and pipes. When used with its identifier argument, this can be used to maintain sync with between applications while other commands are being executed.
  • PreviewPipe.start opens the named pipe. Unlike the TCP based server, only one client can be connected to the pipe at a time.
  • PreviewPipe.stop closes the named pipe.

To start the server, execute previewSocket.listen.

previewSocket.listen port:12378 open:true

Once the port is open, clients can connect to setup a preview renderer and receive renderings.

Protocol

These codes are used to identify any data sent to or from Preview Socket, and are used to prefix the packet.

Preview Socket uses a request/response protocol to communicate with clients. For the most part, this means that the client sends request packets to the Preview Socket server. Preview Socket then sends back acknowledgements or errors. Every message sent by the client will have a matching response from Preview Socket.

	#define	LXfPREVIEWMSG_REQ				0x10000000
	#define	LXfPREVIEWMSG_ACK				0x20000000
 
	#define LXiPREVIEWMSG_ACK_ERROR				(LXfPREVIEWMSG_ACK | 0x01000000)
 
	// Initial Setup
	#define	LXiPREVIEWMSG_REQ_SET_PROTOCOL			(LXfPREVIEWMSG_REQ | 0x00001000)
	#define	LXiPREVIEWMSG_ACK_PROTOCOL_SET			(LXfPREVIEWMSG_ACK | 0x00001000)
 
	// Image Property Setup
	#define	LXiPREVIEWMSG_REQ_SET_RES			(LXfPREVIEWMSG_REQ | 0x00002000)
	#define	LXiPREVIEWMSG_ACK_RES_SET			(LXfPREVIEWMSG_ACK | 0x00002000)
 
	#define	LXiPREVIEWMSG_REQ_SET_FORMAT			(LXfPREVIEWMSG_REQ | 0x00002001)
	#define	LXiPREVIEWMSG_ACK_FORMAT_SET			(LXfPREVIEWMSG_ACK | 0x00002001)
 
	#define	LXiPREVIEWMSG_REQ_SET_STEREO			(LXfPREVIEWMSG_REQ | 0x00002002)
	#define	LXiPREVIEWMSG_ACK_STEREO_SET			(LXfPREVIEWMSG_ACK | 0x00002002)
 
	#define	LXiPREVIEWMSG_REQ_SET_SAMPLES			(LXfPREVIEWMSG_REQ | 0x00002003)
	#define	LXiPREVIEWMSG_ACK_SAMPLES_SET			(LXfPREVIEWMSG_ACK | 0x00002003)
 
	#define	LXiPREVIEWMSG_REQ_SET_DRAFT			(LXfPREVIEWMSG_REQ | 0x00002004)
	#define	LXiPREVIEWMSG_ACK_DRAFT_SET			(LXfPREVIEWMSG_ACK | 0x00002004)
 
	#define	LXiPREVIEWMSG_REQ_SET_FINAL			(LXfPREVIEWMSG_REQ | 0x00002005)
	#define	LXiPREVIEWMSG_ACK_FINAL_SET			(LXfPREVIEWMSG_ACK | 0x00002005)
 
	#define	LXiPREVIEWMSG_REQ_SET_EXTENDED			(LXfPREVIEWMSG_REQ | 0x00002006)
	#define	LXiPREVIEWMSG_ACK_EXTENDED_SET			(LXfPREVIEWMSG_ACK | 0x00002006)
 
	#define	LXiPREVIEWMSG_REQ_SET_EFFECT			(LXfPREVIEWMSG_REQ | 0x00002007)
	#define	LXiPREVIEWMSG_ACK_EFFECT_SET			(LXfPREVIEWMSG_ACK | 0x00002007)
 
	#define	LXiPREVIEWMSG_REQ_SET_MOUSEPOS			(LXfPREVIEWMSG_REQ | 0x00002008)
	#define	LXiPREVIEWMSG_ACK_MOUSEPOS_SET			(LXfPREVIEWMSG_ACK | 0x00002008)
 
	#define	LXiPREVIEWMSG_REQ_SET_CAMERA			(LXfPREVIEWMSG_REQ | 0x00002009)
	#define	LXiPREVIEWMSG_ACK_CAMERA_SET			(LXfPREVIEWMSG_ACK | 0x00002009)
 
	// Frame Requests
	#define	LXiPREVIEWMSG_REQ_SEND_FULL_FRAME		(LXfPREVIEWMSG_REQ | 0x00003000)
	#define	LXiPREVIEWMSG_ACK_SEND_FULL_FRAME		(LXfPREVIEWMSG_ACK | 0x00003000)
 
	#define	LXiPREVIEWMSG_PREVIEW_UNAVAILABLE				     0x00000001
	#define	LXiPREVIEWMSG_PREVIEW_RGBAFP					     0x00000002
	#define	LXiPREVIEWMSG_PREVIEW_COMPLETE					     0x00000100
 
	#define	LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGB24		(LXiPREVIEWMSG_ACK_SEND_FULL_FRAME)
	#define	LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGBAFP	(LXiPREVIEWMSG_ACK_SEND_FULL_FRAME | LXiPREVIEWMSG_PREVIEW_RGBAFP)
	#define	LXiPREVIEWMSG_ACK_FULL_FRAME_UNAVAILABLE	(LXiPREVIEWMSG_ACK_SEND_FULL_FRAME | LXiPREVIEWMSG_PREVIEW_UNAVAILABLE)
	#define	LXiPREVIEWMSG_ACK_FULL_FRAME_DONE		(LXiPREVIEWMSG_ACK_FULL_FRAME_UNAVAILABLE | LXiPREVIEWMSG_PREVIEW_COMPLETE)
 
	// Preview image ID
	#define	LXiPREVIEWMSG_REQ_GET_ID			(LXfPREVIEWMSG_REQ | 0x00004000)
	#define	LXiPREVIEWMSG_ACK_ID_GET			(LXfPREVIEWMSG_ACK | 0x00004000)

In general, REQ_ packets are sent by the client to the server, ACK_ packets are sent from the server to the client in response.

The datatypes used for the packet are as follows. Note that when packing multi-byte datatypes, Preview Socket uses Intel byte order (little endian), as opposed to the Motorola byte order (big endian) more commonly used more with networking.

Shorthand Description C Datatype
U1 One byte unsigned integer unsigned char
U1[] Array of one byte unsigned integers unsigned char []
U4 Four byte unsigned integer unsigned int
Float Four byte floating point number float
Float[] Array of four byte floating point numbers float []
S Array of bytes representing a string char []

Errors

ACK_ERROR

Server to Client: The last request resulted in an error. The request was not completed. This may be sent as a response to any request made by the client in place of an acknowledgement.

Data Description
U4 LXiPREVIEWMSG_ACK_ERROR
U4 LxResult code describing the error
U4 Length of an optional string describing the error (may be a human-readable or internal string); may be 0 if no string is sent.
S If the previous U4 is non-zero, this is an array of bytes represnting a C string describing the error message. Note that the the string is NOT '\0' terminated.


Initialization and Setup

Before doing anything else after connection to the server, the client must indicate which version of the protocol to use. After that, the other the other setup requests can be made at any time to change the render, although some must be called before obtaining the first frame.

REQ_SET_PROTOCOL

Client to Server: Request to set the protocol used to communicate with Preview Socekt. Currently only the default protocol 0 is supported, and attempts to set any other protocol will result in an ACK_ERROR. This message must be sent before any other messages are sent.

Data Description
U4 LXiPREVIEWMSG_REQ_SET_PROTOCOL
U1 Protocol used for communication. Currently only protocol 0 is supported.

ACK_PROTOCOL_SET

Server to Client: Acknowledgement to REQ_SET_PROTOCOL stating that the requested protocol will be used for communication between Preview Socekt and the client.

Data Description
U4 LXiPREVIEWMSG_ACK_PROTOCOL_SET

REQ_SET_RES

Client to Server: Request to the Preview Socket to set the frame resolution. The server will begin rendering when the resolution changes. The resolution must be set before frames can be sent.

Data Description
U4 LXiPREVIEWMSG_REQ_SET_RES
U4 width in pixels
U4 height in pixels

ACK_RES_SET

Server to Client: Acknowledgement to a REQ_SET_RES event stating that the resolution has been set.

Data Description
U4 LXiPREVIEWMSG_ACK_RES_SET

REQ_SET_FORMAT

Client to Server: Request to the Preview Socket to set the frame's pixel format. The server will begin rendering when the format changes. If the format is not set, LXiIMD_BYTE is assumed. Note that the only options are byte RGB and float RGBA.

Data Description
U4 LXiPREVIEWMSG_REQ_SET_FORMAT
U4 Format; LXiIMD_BYTE for 24 bit RGB; LXiIMD_FLOAT for floating point RGBA. These defines can be found in the image server documentation.

ACK_FORMAT_SET

Server to Client: Acknowledgement to a REQ_SET_FORMAT event stating that the format has been set.

Data Description
U4 LXiPREVIEWMSG_ACK_FORMAT_SET

REQ_SET_STEREO

Client to Server: Request to the Preview Socket to set the stereoscopic mode, which causes it to render frames for the left eye, the right eye or monocularly. By simultaneously connecting to two Preview Sockets set to different eyes, you can render a stereo pair. Preview Socket will begin rendering when the mode changes. If no stereoscopic mode is set, monocular mode is assumed.

Data Description
U4 LXiPREVIEWMSG_REQ_SET_STEREO
U4 Mode; 0 for monocular, 1 for the left eye and 2 for the right eye

ACK_STEREO_SET

Server to Client: Acknowledgement to a REQ_SET_STEREO event stating that the stereo mode has been set.

Data Description
U4 LXiPREVIEWMSG_ACK_STEREO_SET

REQ_SET_SAMPLES

Client to Server: Request to Preview Socekt to set the maximum number of samples. Only applies when preview quality is DRAFT or EXTENDED. If the quality is FINAL, the settings from the scene are used. Preview Socket will begin rendering when the number of samples changes. If not explicitly set, the current scene values are used.

Data Description
U4 LXiPREVIEWMSG_REQ_SET_SAMPLES
U4 Samples; maximum number of samples to use

ACK_SAMPLES_SET

Server to Client: Acknowledgement to a REQ_SET_SAMPLES event stating that the number of samples have been set.

Data Description
U4 LXiPREVIEWMSG_ACK_SAMPLES_SET

REQ_SET_DRAFT

Client to Server: Request that Preview Socket use draft quality with an optional number of antialiasing samples and quality level (fraction of final quality). Preview Socket will begin rendering when the quailty changes. If the samples or quality level are set to 0, the current preferences are used.

Data Description
U4 LXiPREVIEWMSG_REQ_SET_DRAFT
U4 Samples; maximum number of samples to use
Float Quality; fraction of final render (0.0 to 1.0)

ACK_DRAFT_SET

Server to Client: Acknowledgement to a REQ_SET_DRAFT event stating that the number of samples have been set.

Data Description
U4 LXiPREVIEWMSG_ACK_DRAFT_SET

REQ_SET_FINAL

Client to Server: Request final render quality Preview. Preview will begin rendering when the quailty changes. The current scene and rendering preferences are used.

Data Description
U4 LXiPREVIEWMSG_REQ_SET_FINAL

ACK_FINAL_SET

Server to Client: Acknowledgement to a REQ_SET_FINAL event stating that the number of samples have been set.

Data Description
U4 LXiPREVIEWMSG_ACK_FINAL_SET

REQ_SET_EXTENDED

Client to Server: Request extended quality for the Preview Socket render, using an optional number of antialiasing samples. Preview Socket will begin rendering when the quality changes. Rendering settings are the same as used for final rendering quality, except using a potentially greater number of antialiasing samples. If the samples are set to 0, the current preferences are used.

Data Description
U4 LXiPREVIEWMSG_REQ_SET_EXTENDED
U4 Samples; maximum number of samples to use

ACK_EXTENDED_SET

Server to Client: Acknowledgement to a REQ_SET_EXTENDED event stating that the number of samples have been set.

Data Description
U4 LXiPREVIEWMSG_ACK_EXTENDED_SET

REQ_SET_EFFECT

Client to Server: Request to Preview Socket to set the output channel or effect. Preview will begin rendering when the effect changes.

Data Description
U4 LXiPREVIEWMSG_REQ_SET_EXTENDED
U4 length of effect string (without a '\0')
S Effect string (not terminated with '\0')

ACK_EFFECT_SET

Server to Client: Acknowledgement to a REQ_SET_EFFECT event stating that the effect has been set.

Data Description
U4 LXiPREVIEWMSG_ACK_EFFECT_SET

REQ_SET_MOUSEPOS

Client to Server: Request to Preview Socket to focus the preview on the mouse position -- or rather, the given X/Y position within the rendering frame as defined by a previous call to REQ_SET_RES. Preview will continue rendering, but reorder the buckets to concentrate on the position specified for the next update only, You must send continue to send repeated mouse position requests for continued refinement.

Data Description
U4 LXiPREVIEWMSG_REQ_SET_MOUSEPOS
U4 X position of mouse
U4 Y position of mouse

ACK_MOUSEPOS_SET

Server to Client: Acknowledgement to a REQ_SET_MOUSEPOS event stating that the mouse position has been set.

Data Description
U4 LXiPREVIEWMSG_ACK_MOUSEPOS_SET

REQ_SET_CAMERA

Client to Server: Request to set camera position, direction & zoom for preview. Preview Socket will restart rendering with the new camera state.

Data Description
U4 LXiPREVIEWMSG_REQ_SET_CAMERA
Float[] XYZ position of camera
Float[] XYZ direction of camera
Float Zoom factor

ACK_CAMERA_SET

Server to Client: Acknowledgement to a REQ_SET_CAMERA event stating that the mouse position has been set.

Data Description
U4 LXiPREVIEWMSG_ACK_CAMERA_SET

Requesting Rendered Frames

To request a rendered frame, you send the LXiPREVIEWMSG_REQ_SEND_FULL_FRAME message. The client will receive the currently waiting RGB or RGBAFP if one is available. Otherwise it will get ACK_FULL_FRAME_UNAVAILABLE if there is not currently a frame ready to send, or LXiPREVIEWMSG_ACK_FULL_FRAME_DONE if rendering has finished, thus indicating that the most recently received frame represents the final render.

REQ_SEND_FULL_FRAME

Client to Server: Request the most recently rendered image as single large packet.

Data Description
U4 LXiPREVIEWMSG_REQ_SEND_FULL_FRAME

ACK_SEND_FULL_FRAME_RGB24

Server to Client: Send the most recently rendered frame to the client as a response to REQ_SEND_FULL_FRAME. The frame is sent as a single large packet containing a byte-format RGB image. The RGB part of the packet is width * height * 3 bytes long.

Data Description
U4 LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGB24 (synonym: LXiPREVIEWMSG_ACK_SEND_FULL_FRAME) U4 Image width U4 Image height U1[] Byte array of RGB color triples

ACK_SEND_FULL_FRAME_RGBAFP

Server to Client: Send the most recently rendered floating point frame to the client as a response to REQ_SEND_FULL_FRAME, when the format is LXiIMD_FLOAT. The frame is sent as a single large packet. The RBGA part of the packet is width * height * 16 bytes long (4 floats per pixel).

Data Description
U4 LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGBAFP U4 Image width U4 Image height Float[] Float array of RGBA colors

ACK_FULL_FRAME_UNAVAILABLE

Server to Client: Sent as an alternate response to REQ_SEND_FULL_FRAME_RGB24 and REQ_SEND_FULL_FRAME_RGBAFP when no frame is available to send.

Data Description
U4 LXiPREVIEWMSG_ACK_FULL_FRAME_UNAVAILABLE

ACK_FULL_FRAME_DONE

Client to Server: Sent as an alternate request to REQ_SEND_FULL_FRAME_RGB24 and REQ_SEND_FULL_FRAME_RGBAFP when the render is complete and no more rendering is required. The last frame sent to the client represents the most recent render, and no further renders will be sent until the scene or Preview Socket state are changed.

Data Description
U4 LXiPREVIEWMSG_ACK_FULL_FRAME_DONE

Render ID

The render ID is a unique identifier assigned to each specific requested frame. This can be used to synchronize the client and server through Preview.suspend

REQ_GET_ID

Client to Server: Request the server ID assigned to the last completed render. This message is only valid after REQ_SEND_FULL_FRAME has returned a state, including FULL_FRAME_DONE. This ID can be used for synchronizing the client and server with Preview.suspend.

Data Description
U4 LXiPREVIEWMSG_REQ_GET_ID

ACK_ID_GET

Server to Client: Acknowledgement to a REQ_GET_ID event, returning application ID assigned to the last completed render. If the last REQ_SEND_FULL_FRAME did not return a state (which may include FULL_FRAME_DONE), this value will be 0.

Data Description
U4 LXiPREVIEWMSG_ACK_ID_GET
U4 Render ID

Basic Loop

A client implements the protocol by first connecting to the server, and then sending the LXiPREVIEWMSG_REQ_SET_PROTOCOL packet to indicate which version of the protocol it wants to communicate over. The current protocol version is 0. An acknowledgment will be returned indicating that the protocol was set.

Once the protocol version has been established, the client can send other setup-oriented requests to change the resolution, pixel format, samples, quality, stereo eye, camera position, and other options. These can be set at any time, although some must be set at least once before rendering can begin. After sending any request, the client should wait for an acknowledgment to ensure that the request was accepted and that there were no errors.

Once setup is complete, the client can begin asking for frames. Preview Socket must be polled for the latest frame; it will not simply send frames to the client without being asked first. Sending LXiPREVIEWMSG_REQ_SEND_FULL_FRAME to the server will return either LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGB24 or LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGBAFP depending on the pixel format previously defined along with the most recent render. If there is no render, the response will be LXiPREVIEWMSG_ACK_FULL_FRAME_UNAVAILABLE to indicate that no newer frame is available, or LXiPREVIEWMSG_ACK_FULL_FRAME_DONE to indicate that the last render received by the client represents the final render.

Note that if a packet header cannot be identified by Preview Socket, the connection will be closed. This is because there is no way for Preview Socket to know how far to seek ahead to find the next packet, so there's really nothing else it can do.

Synchronizing

Often a Preview Socket client will also use Telnet to manipulate the scene through commands. The client usually isn't interested in any new renders until it is done sending all of its commands. To ensure this, it first sends the Preview.suspend command over telnet to suspend all Preview Sockets.

preview.suspend true

At this point there may already be a pending image waiting to be sent to the client. The client isn't interested in this image, and wants to be sure that it gets only images after it has finished executing its commands. This can be done by using LXiPREVIEWMSG_REQ_GET_ID to request the unique identifier of the last image sent to the client. When all other commands have been sent over telnet, the client can resume rendering with preivew.suspend, but this time setting the suspend argument to false and providing the previously-obtained unique identifier.

preview.suspend false identifier

The identifier is optional, in which case the client's next request for an image may actually be a pending image from before Preview Socket was suspended. This behavior is useful if you just wanted to pause rendering to free up some CPU time, and then want it to continue afterward.

Headless

When using Preview Socket with headless mode, you must launch modo with the -telnet: switch and a port number to start the telnet server. Chances are that you'll be doing this anyway, since you'll need be using telnet to communicate with modo to control manipulate the scene. If you do not launch the application with -telnet:, Preview Socket will fail to process input and appear to hang. This happens because modo's main loop is blocked while it waits for the user to enter a new command in normal headless mode. Using the -telnet: switch causes the application to use an alternate input loop for processing network traffic, and thus allows Preview Socket to work properly.

More Information