This Xtra will only work on Windows computers.

Your computer must be connected to a source of video by an internal socket, third party video digitizing board, USB or web cam. The software uses VFW to access the video device. Use the attached application Vidmem.exe to test if your video is properly attached and installed. Also try changing sizes and bit depths to determine the capabilities of your setup. The Xtra does not work with DV cameras through FireWire

The Xtra lets you display the video on stage, Grab a video frame to a cast member and get the color value of pixels in the video. And Mask portions of the video

If you have suggestions, problems or comments about this Xtra you can write to :

info@smoothware.com

To find information about this Xtra and other Xtras, or to purchase this or another Xtra visit URL

www.smoothware.com

Put the Xtra in the "Xtras" folder near your Director application. Then call -

set temp = Xtra "TrackThemColors"

set MaskObj = new(temp)

Have MaskObj declared as a Global variable since you will be referring to it later.

You will probably be doing this in the beginning of the movie so your handler will probably look like this:

on StartMovie

Global MaskObj

if not MaskObj then

set temp = Xtra "TrackThemColors"

set MaskObj = new(temp)

end if

end StartMovie

When you are done you should close the Xtra by assigning 0 to the variable.

on StopMovie

global MaskObj

set MaskObj = 0

end

Starting the video flow.

In order to access the video stream, the Xtra has to establish a connection with the video capture device and allocate memory. None of the other methods in this Xtra will work without calling the InitVideo() first. You should call the InitVideo() method only once, and be sure to call CleanUp() before you leave.

The InitVideo(MaskObj, Width, height, source) method takes four arguments:

MaskObj - The instance number returned from New().

Width - defines the width in Pixels of the video frame you want.

Height - defines the height in Pixels of the video frame you want.

source - defines the video digitizer that will be used. In most cases use 1. If you have more than one video digitizers installed in your computer . If you specify a number higher than the number of sources in your computer , the Init will fail

The InitVideo () method returns a list with three items [Width, Height, bytesPerPixel]:

Width - returns the actual width in pixels of the video frame.

Height - returns the actual height in pixels of the video frame.

bytesPerPixel - returns the bit depth of the video capture (should be either 24 or 32)

Note. On Windows, different video boards and driver have different behavior. the size and bit depth they operate in varies. It is important to check the return value of this method and see that you have actually received the size you have requested. Often you will not.

Once you are done tracking, you should call the CleanUp() method. Do not attempt to call InitVideo() again without calling the CleanUp() first.

The CleanUp(MaskObj) method takes one argument.

MaskObj - The instance number returned from New().

To open the standard Video Settings dialog boxes which allow you to change the video source or change the parameters for your camera, use the VideoSettings() method. This is very useful for changing the brightness, saturation and contrast of the video to create better conditions for the tracking, always use 24 RGB or 32 RGBA

The VideoSettings(MaskObj, whichDialog) takes two arguments.

MaskObj - The instance number returned from New().

whichDialog - 1 opens the format dialog box , 2- source dialog box. 3 - display dialog box. 4- compression dialog box

The VideoSettings () method returns a list with three items [Width, Height, bytesPerPixel]:

Width - returns the actual width in pixels of the video frame.

Height - returns the actual height in pixels of the video frame.

bytesPerPixel - returns the bit depth of the video capture (should be either 24 or 32)

it is important that you examine the return values of this method and see that the changes you requested have actually been applied. Not all dialogs available on some drivers

To View the video source on the screen you should call ShowVideo()

The ShowVideo(MaskObj) method takes three arguments.

MaskObj - The instance number returned from New().

sourceRect - A rect defining he area in the frame you want to show, this allows you to select a portion of the frame to be displayed.

destRect - A rect defining the area on stage where you want the video shown. This allows you to stretch the video.

Please note that the ShowVideo() method should be called repeatedly, to get a continuously changing display of the video source. Also note that ShowVideo is "direct to stage", if you wish to have sprites overlaid on top of the video you should use GrabToCast()

Grabbing a video frame to a cast member

You can use the Xtra to grab frames from the video to a cast member. This is useful for creating background images for visualizing the tracking and for other purposes.

To grab a video frame to a cast member use the GrabToCast() method.

The GrabToCast (MaskObj, CastMemberNum) takes two parameters.

MaskObj - The instance number returned from New().

CastMemberNum - The number of the cast member to which you wish to grab the frame. (Have an Image cast member waiting in that location)

Getting a hold of the pixels in the video:

Sometimes you might want to get a hold of the actual pixels of the video frame. This is useful for analyzing the colors and for creating any other tracking method that is not supplied by the Xtra. Getting the pixels into director is surprisingly fast, and you should not hesitate to ask for thousands of points at a time. The two methods that send back pixel colors are GetColors() and GetAllColors().

The GetColors() method lets you define the points you are interested in and returns the colors of those points.

The GetColors(MaskObj, points) takes two arguments:

MaskObj - The instance number returned from New().

points - A list of points to be returned [point1, point2, point3...]

The GetColors() method returns a list with the following sequence : [red1, green1, blue1, horizontal1, vertical1, red2, green2, blue2, horizontal2, vertical2...]

red -the red component of the pixel 0-255.

green -the green component of the pixel 0-255.

blue -the blue component of the pixel 0-255.

horizontal - the horizontal location of the pixel - this is an echo of the argument you sent as a point .

vertical - the vertical location of the pixel - this is an echo of the argument you sent as a point .

The GetAllColors lets get a large amount of pixels fast. You define an area and the Xtra returns all the pixels in that area. This method is faster than GetColors() but the results are not formatted as conveniently.

The GetAllColors(MaskObj, area) takes two arguments:

MaskObj - The instance number returned from New().

area - a rect of the area of interest.

The GetAllColors() method returns a list of colors in the following sequence [red1, green1, blue1, red2, green2, blue2, red3, green3, blue3...]

red -the red component of the pixel 0-255.

green -the green component of the pixel 0-255.

blue -the blue component of the pixel 0-255.

using these results is not that straight forward.

Say you want to receive all the pixels from an area at the upper left corner of the video . Say the rect you are defining is rect(0,0,100,100).

Therefore your call will look like this :

put GetAllColors(MaskObj, rect(0,0,100,100)) into TheResult

therefor the Xtra will return a list with 3 X 100 x 100 =30,000 items

Than say you want to see the color of pixel 10 - horizontal , 50 - vertical .

Set TheWidth = 100 -- remember we asked for a 100*100 rect

set PixelH = 10 -- our pixels horizontal location

Set PixelV = 50 -- our pixels vertical location

Set OurPixel = PixelH *3 + PixelV *TheWidth

set red =theresult [OurPixel]

set green =theresult [OurPixel+1]

set blue =theresult [OurPixel+2]

This looks complicated but you only have to figure it out once...

Getting a hold of the brightness of pixels in the video:

Similar to the methods that return pixel colors, but these return the brightness instead . The two methods that send back pixel brightness are GetGrays() and GetAllGrays().

The GetGrays() method lets you define the points you are interested in and returns the colors of those points.

The GetGrays(MaskObj, points) takes two arguments:

MaskObj - The instance number returned from New().

points - A list of points to be returned [point1, point2, point3...]

The GetGrays() method returns a list with the following sequence : [brightness1, horizontal1, vertical1, brightness2, horizontal2, vertical2...]

brightness -the average gray level of the pixel 0-255.

horizontal - the horizontal location of the pixel - this is an echo of the argument you sent as a point .

vertical - the vertical location of the pixel - this is an echo of the argument you sent as a point .

The GetAllGrays lets get a large amount of pixels fast. You define an area and the Xtra returns all the pixels in that area. This method is faster than GetGrays() but the results are not formatted as conveniently.

The GetAllGrays(MaskObj, area) takes two arguments:

MaskObj - The instance number returned from New().

area - a rect of the area of interest.

The GetAllGrays() method returns a list of brightnesses in the following sequence [brightness1,brightness2, brightness3...] the number of items will be the width of the area of interest X the height of the area of interest.

brightness -the average gray level of the pixel 0-255.

To mask a specific color of the video (blue screen-chroma key) use the maskRemoveColor() method. This method will create a mask over all the areas of the video that are similar enough to the color you specify. Se the other mask methods to learn how to specify the fill of the mask and foreground and how to edit the mask and view it. Note that you must call maskRemoveColor() repeatedly with every new frame . maskRemoveColor(MaskObj,error,R,G,B) takes 5 arguments:

MaskObj - The instance number returned from New().

Error — a number between 0 — 442 that defines how similar a pixel needs to be to the specified color in order to be included in the mask

R,G,B — the color values of the color to be removed. It is useful to get these colors from GetColors() by pointing on the frame.

To remove a reference background use the maskRemoveBackground() method . This method will create a mask over all the areas of the video that are similar enough to the corresponding pixels in the reference frame, you must call maskSetBackground once before calling this method to establish the background. See the other mask methods to learn how to specify the fill of the mask and foreground, and how to edit the mask and view it. Note that you must call maskRemoveColor() repeatedly with every new frame .

maskRemoveColor(MaskObj,error) takes 2 arguments:

MaskObj - The instance number returned from New().

Error — a number between 0 — 442 that defines how similar a pixel needs to be to the coresponding pixel in the reference frame in order to be included in the mask

You create a background reference frame to be used with maskRemoveBackground with the maskSetBackground() method.

The maskSetBackground(MaskObj) method takes 1 argument:

MaskObj - The instance number returned from New().

To view the mask results on stage you must call the maskShowMask () method. The maskShowMask method is similar to the ShowVideo method but shows the masked video rather than the original video. You must call either maskRemoveBackground or maskRemoveColor prior to calling this method. The maskShowMask (MaskObj, sourceRect, destRect) takes 3 arguments:

MaskObj - The instance number returned from New().

sourceRect - A rect defining he area in the frame you want to show, this allows you to select a portion of the frame to be displayed.

destRect - A rect defining the area on stage where you want the video shown. This allows you to stretch the video.

Please note that the maskShowMask() method should be called repeatedly, to get a continuously changing display of the mask results. Also note that maskShowMask is "direct to stage", if you wish to have sprites overlaid on top of the video you should use maskGrabToCast()

You can use the Xtra to grab frames from the masked video to a cast member. This is useful wqhen you want to place it on stage as a sprite with other sprites over it.

To grab a masked video frame to a cast member use the maskGrabToCast() method.

ThemaskGrabToCast (MaskObj, CastMemberNum) takes two parameters.

MaskObj - The instance number returned from New().

CastMemberNum - The number of the cast member to which you wish to grab the masked frame. (Have a 32 bit Image cast member waiting in that location)

To set the fill of the masked areas of the video, use the maskSetMaskFill() method. Note that you should only call thius method once when you want to change the fill, and that will take affect until you change it again. The default fill for the mask is solid black.

The maskSetMaskFill(MaskObj, fillParamsList) takes two arguments :

MaskObj - The instance number returned from New().

FillParamsList — a list containing the following:

0 — Set the fill of the mask to the live video . [0]

1 — Set the fill of the mask to a solid color , then it is followed by R,G,B specifying the color to use. [1,255,0,0] use solid red

2 — Set the fill of the mask to the background frame set with maskSetBackground [2]

3 — Set the fill of the mask to a member, then it is followed by the number of the member to use. [3,42] Note that a member the size of the video and in 32 bit must be waiting in that position.

To set the fill of the masked areas of the video, use the maskSetForegroundFill() method. Note that you should only call this method once when you want to change the fill, and that will take affect until you change it again. The default fill for the mask is live video.

The maskSetForegroundFill (MaskObj, fillParamsList) takes two arguments :

MaskObj - The instance number returned from New().

FillParamsList — a list containing the following:

0 — Set the fill of the foreground to the live video . [0]

1 — Set the fill of the foreground to a solid color , then it is followed by R,G,B specifying the color to use. [1,255,0,0] use solid red

2 — Set the fill of the foreground to the background frame set with maskSetBackground [2]

3 — Set the fill of the foreground to a member, then it is followed by the number of the member to use. [3,42] use member 42. Note that a member the size of the video and in 32 bit must be waiting in that position.

To close little holes in the mask and foreground that are usually the result of noise in the video use maskEditCloseHoles() . Note that you should only call this method once when you want to change the settingsl, and that will take affect until you change it again. The default is 0 . The maskEditCloseHoles(MaskObj, howBig) method takes 2 arguments :

MaskObj - The instance number returned from New().

HowBig — a number between 0 — 8 that describes what size holes will be filled. 0 = no filling.

To enlarge or shrink the size of the masked areas use maskEditGrowShrink() . Note that you should only call this method once when you want to change the settings, and that will take affect until you change it again. The default is 0 . The maskEditGrowShrink (MaskObj, howMuch) method takes 2 arguments :

MaskObj - The instance number returned from New().

HowMuch — a number between -10 — 10 that describes by how much (in pixels) the areas of the mask should be enlarged. Negative numbers mean shrink.

To create an outline around the masked areas use the maskEditMakeOutline() method. Note that you should only call this method once when you want to change the settings, and that will take affect until you change it again. The default is 0 . The maskEditMakeOutline (MaskObj, howWide) method takes 2 arguments :

MaskObj - The instance number returned from New().

HowWide — a number between 0 — 10 that describes the width of the outline

You can ask the Xtra to soften the edges of the mask. This creates an antialiased effect where the border pixels are a blend of the foreground and mask fills. To achieve this effect use the maskEditSoftEdge() method. Note that you should only call this method once when you want to change the settings, and that will take affect until you change it again. The default is 0 . The maskEditSoftEdge (MaskObj, yesOrNo) method takes 2 arguments :

MaskObj - The instance number returned from New().

yesOrNo — 1 or 0 . 1- soften edges. 0 — don’t.

To flip the video horizontally and or vertically call the Flip() method. Note that you need to call this method once. Note that this actually flips the video frame and all resulting tracking and masking will be flipped. The Flip (MaskObj, horiz, vert) method takers 3 arguments:

MaskObj - The instance number returned from New().

Horiz — 1 flip horizontally , 0 - don’t.

vert — 1 flip vertically , 0 - don’t.