ai/lib
AIErrorCode
Defined in: ai/lib/types/error.ts:65
Standardized error codes that AIError can have.
API_NOT_ENABLED
API_NOT_ENABLED:
"api-not-enabled"
Defined in: ai/lib/types/error.ts:82
An error due to the Firebase API not being enabled in the Console.
ERROR
ERROR:
"error"
Defined in: ai/lib/types/error.ts:67
A generic error occurred.
FETCH_ERROR
FETCH_ERROR:
"fetch-error"
Defined in: ai/lib/types/error.ts:76
An error occurred while performing a fetch.
INVALID_CONTENT
INVALID_CONTENT:
"invalid-content"
Defined in: ai/lib/types/error.ts:79
An error associated with a Content object.
INVALID_SCHEMA
INVALID_SCHEMA:
"invalid-schema"
Defined in: ai/lib/types/error.ts:85
An error due to invalid Schema input.
NO_API_KEY
NO_API_KEY:
"no-api-key"
Defined in: ai/lib/types/error.ts:88
An error occurred due to a missing Firebase API key.
NO_APP_ID
NO_APP_ID:
"no-app-id"
Defined in: ai/lib/types/error.ts:91
An error occurred due to a missing Firebase app ID.
NO_MODEL
NO_MODEL:
"no-model"
Defined in: ai/lib/types/error.ts:94
An error occurred due to a model name not being specified during initialization.
NO_PROJECT_ID
NO_PROJECT_ID:
"no-project-id"
Defined in: ai/lib/types/error.ts:97
An error occurred due to a missing project ID.
PARSE_FAILED
PARSE_FAILED:
"parse-failed"
Defined in: ai/lib/types/error.ts:100
An error occurred while parsing.
REQUEST_ERROR
REQUEST_ERROR:
"request-error"
Defined in: ai/lib/types/error.ts:70
An error occurred in a request.
RESPONSE_ERROR
RESPONSE_ERROR:
"response-error"
Defined in: ai/lib/types/error.ts:73
An error occurred in a response.
SESSION_CLOSED
SESSION_CLOSED:
"session-closed"
Defined in: ai/lib/types/error.ts:106
An error occurred due to a session being closed.
UNSUPPORTED
UNSUPPORTED:
"unsupported"
Defined in: ai/lib/types/error.ts:103
An error occurred due an attempt to use an unsupported feature.
BlockReason
Defined in: ai/lib/types/enums.ts:142
Reason that a prompt was blocked.
BLOCKLIST
BLOCKLIST:
"BLOCKLIST"
Defined in: ai/lib/types/enums.ts:154
Content was blocked because it contained terms from the terminology blocklist.
OTHER
OTHER:
"OTHER"
Defined in: ai/lib/types/enums.ts:150
Content was blocked, but the reason is uncategorized.
PROHIBITED_CONTENT
PROHIBITED_CONTENT:
"PROHIBITED_CONTENT"
Defined in: ai/lib/types/enums.ts:158
Content was blocked due to prohibited content.
SAFETY
SAFETY:
"SAFETY"
Defined in: ai/lib/types/enums.ts:146
Content was blocked by safety settings.
FinishReason
Defined in: ai/lib/types/enums.ts:165
Reason that a candidate finished.
BLOCKLIST
BLOCKLIST:
"BLOCKLIST"
Defined in: ai/lib/types/enums.ts:189
The candidate content contained forbidden terms.
MALFORMED_FUNCTION_CALL
MALFORMED_FUNCTION_CALL:
"MALFORMED_FUNCTION_CALL"
Defined in: ai/lib/types/enums.ts:201
The function call generated by the model was invalid.
MAX_TOKENS
MAX_TOKENS:
"MAX_TOKENS"
Defined in: ai/lib/types/enums.ts:173
The maximum number of tokens as specified in the request was reached.
OTHER
OTHER:
"OTHER"
Defined in: ai/lib/types/enums.ts:185
Unknown reason.
PROHIBITED_CONTENT
PROHIBITED_CONTENT:
"PROHIBITED_CONTENT"
Defined in: ai/lib/types/enums.ts:193
The candidate content potentially contained prohibited content.
RECITATION
RECITATION:
"RECITATION"
Defined in: ai/lib/types/enums.ts:181
The candidate content was flagged for recitation reasons.
SAFETY
SAFETY:
"SAFETY"
Defined in: ai/lib/types/enums.ts:177
The candidate content was flagged for safety reasons.
SPII
SPII:
"SPII"
Defined in: ai/lib/types/enums.ts:197
The candidate content potentially contained Sensitive Personally Identifiable Information (SPII).
STOP
STOP:
"STOP"
Defined in: ai/lib/types/enums.ts:169
Natural stop point of the model or provided stop sequence.
FunctionCallingMode
Defined in: ai/lib/types/enums.ts:209
Function calling mode for the model.
ANY
ANY:
"ANY"
Defined in: ai/lib/types/enums.ts:221
Model is constrained to always predicting a function call only.
If allowed_function_names is set, the predicted function call will be
limited to any one of allowed_function_names, else the predicted
function call will be any one of the provided function_declarations.
AUTO
AUTO:
"AUTO"
Defined in: ai/lib/types/enums.ts:214
Default model behavior; model decides to predict either a function call or a natural language response.
NONE
NONE:
"NONE"
Defined in: ai/lib/types/enums.ts:226
Model will not predict any function call. Model behavior is same as when not passing any function declarations.
HarmBlockMethod
Defined in: ai/lib/types/enums.ts:74
This property is not supported in the Gemini Developer API (GoogleAIBackend).
PROBABILITY
PROBABILITY:
"PROBABILITY"
Defined in: ai/lib/types/enums.ts:82
The harm block method uses the probability score.
SEVERITY
SEVERITY:
"SEVERITY"
Defined in: ai/lib/types/enums.ts:78
The harm block method uses both probability and severity scores.
HarmBlockThreshold
Defined in: ai/lib/types/enums.ts:45
Threshold above which a prompt or candidate will be blocked.
BLOCK_LOW_AND_ABOVE
BLOCK_LOW_AND_ABOVE:
"BLOCK_LOW_AND_ABOVE"
Defined in: ai/lib/types/enums.ts:49
Content with NEGLIGIBLE will be allowed.
BLOCK_MEDIUM_AND_ABOVE
BLOCK_MEDIUM_AND_ABOVE:
"BLOCK_MEDIUM_AND_ABOVE"
Defined in: ai/lib/types/enums.ts:53
Content with NEGLIGIBLE and LOW will be allowed.
BLOCK_NONE
BLOCK_NONE:
"BLOCK_NONE"
Defined in: ai/lib/types/enums.ts:61
All content will be allowed.
BLOCK_ONLY_HIGH
BLOCK_ONLY_HIGH:
"BLOCK_ONLY_HIGH"
Defined in: ai/lib/types/enums.ts:57
Content with NEGLIGIBLE, LOW, and MEDIUM will be allowed.
OFF
OFF:
"OFF"
Defined in: ai/lib/types/enums.ts:66
All content will be allowed. This is the same as BLOCK_NONE, but the metadata corresponding
to the HarmCategory will not be present in the response.
HarmCategory
Defined in: ai/lib/types/enums.ts:34
Harm categories that would cause prompts or candidates to be blocked.
HARM_CATEGORY_DANGEROUS_CONTENT
HARM_CATEGORY_DANGEROUS_CONTENT:
"HARM_CATEGORY_DANGEROUS_CONTENT"
Defined in: ai/lib/types/enums.ts:38
HARM_CATEGORY_HARASSMENT
HARM_CATEGORY_HARASSMENT:
"HARM_CATEGORY_HARASSMENT"
Defined in: ai/lib/types/enums.ts:37
HARM_CATEGORY_HATE_SPEECH
HARM_CATEGORY_HATE_SPEECH:
"HARM_CATEGORY_HATE_SPEECH"
Defined in: ai/lib/types/enums.ts:35
HARM_CATEGORY_SEXUALLY_EXPLICIT
HARM_CATEGORY_SEXUALLY_EXPLICIT:
"HARM_CATEGORY_SEXUALLY_EXPLICIT"
Defined in: ai/lib/types/enums.ts:36
HarmProbability
Defined in: ai/lib/types/enums.ts:89
Probability that a prompt or candidate matches a harm category.
HIGH
HIGH:
"HIGH"
Defined in: ai/lib/types/enums.ts:105
Content has a high chance of being unsafe.
LOW
LOW:
"LOW"
Defined in: ai/lib/types/enums.ts:97
Content has a low chance of being unsafe.
MEDIUM
MEDIUM:
"MEDIUM"
Defined in: ai/lib/types/enums.ts:101
Content has a medium chance of being unsafe.
NEGLIGIBLE
NEGLIGIBLE:
"NEGLIGIBLE"
Defined in: ai/lib/types/enums.ts:93
Content has a negligible chance of being unsafe.
HarmSeverity
Defined in: ai/lib/types/enums.ts:112
Harm severity levels.
HARM_SEVERITY_HIGH
HARM_SEVERITY_HIGH:
"HARM_SEVERITY_HIGH"
Defined in: ai/lib/types/enums.ts:128
High level of harm severity.
HARM_SEVERITY_LOW
HARM_SEVERITY_LOW:
"HARM_SEVERITY_LOW"
Defined in: ai/lib/types/enums.ts:120
Low level of harm severity.
HARM_SEVERITY_MEDIUM
HARM_SEVERITY_MEDIUM:
"HARM_SEVERITY_MEDIUM"
Defined in: ai/lib/types/enums.ts:124
Medium level of harm severity.
HARM_SEVERITY_NEGLIGIBLE
HARM_SEVERITY_NEGLIGIBLE:
"HARM_SEVERITY_NEGLIGIBLE"
Defined in: ai/lib/types/enums.ts:116
Negligible level of harm severity.
HARM_SEVERITY_UNSUPPORTED
HARM_SEVERITY_UNSUPPORTED:
"HARM_SEVERITY_UNSUPPORTED"
Defined in: ai/lib/types/enums.ts:135
Harm severity is not supported.
Modality
Defined in: ai/lib/types/enums.ts:233
Content part modality.
AUDIO
AUDIO:
"AUDIO"
Defined in: ai/lib/types/enums.ts:253
Audio.
DOCUMENT
DOCUMENT:
"DOCUMENT"
Defined in: ai/lib/types/enums.ts:257
Document (for example, PDF).
IMAGE
IMAGE:
"IMAGE"
Defined in: ai/lib/types/enums.ts:245
Image.
MODALITY_UNSPECIFIED
MODALITY_UNSPECIFIED:
"MODALITY_UNSPECIFIED"
Defined in: ai/lib/types/enums.ts:237
Unspecified modality.
TEXT
TEXT:
"TEXT"
Defined in: ai/lib/types/enums.ts:241
Plain text.
VIDEO
VIDEO:
"VIDEO"
Defined in: ai/lib/types/enums.ts:249
Video.
SchemaType
Defined in: ai/lib/types/schema.ts:24
Contains the list of OpenAPI data types as defined by the OpenAPI specification
ARRAY
ARRAY:
"array"
Defined in: ai/lib/types/schema.ts:34
Array type.
BOOLEAN
BOOLEAN:
"boolean"
Defined in: ai/lib/types/schema.ts:32
Boolean type.
INTEGER
INTEGER:
"integer"
Defined in: ai/lib/types/schema.ts:30
Integer type.
NUMBER
NUMBER:
"number"
Defined in: ai/lib/types/schema.ts:28
Number type.
OBJECT
OBJECT:
"object"
Defined in: ai/lib/types/schema.ts:36
Object type.
STRING
STRING:
"string"
Defined in: ai/lib/types/schema.ts:26
String type.
AIError
Defined in: ai/lib/errors.ts:27
Error class for the Vertex AI in Firebase SDK.
Constructor
new AIError(
code,message,customErrorData?):AIError
Defined in: ai/lib/errors.ts:35
Constructs a new instance of the AIError class.
code
The error code from AIErrorCode.
code
readonlycode:AIErrorCode
Defined in: ai/lib/errors.ts:36
The error code from AIErrorCode.
customErrorData?
readonlyoptionalcustomErrorData:CustomErrorData
Defined in: ai/lib/errors.ts:38
Optional error data.
abstract AIModel
Defined in: ai/lib/models/ai-model.ts:30
Base class for Firebase AI model APIs.
Instances of this class are associated with a specific Firebase AI Backend and provide methods for interacting with the configured generative model.
Constructor
protectednew AIModel(ai,modelName):AIModel
Defined in: ai/lib/models/ai-model.ts:59
Internal
Constructs a new instance of the AIModel class.
This constructor should only be called from subclasses that provide a model API.
modelName
string
The name of the model being used. It can be in one of the following formats:
my-model(short name, will resolve topublishers/google/models/my-model)models/my-model(will resolve topublishers/google/models/my-model)publishers/my-publisher/models/my-model(fully qualified model name)
_apiSettings
protected_apiSettings:ApiSettings
Defined in: ai/lib/models/ai-model.ts:40
Internal
model
readonlymodel:string
Defined in: ai/lib/models/ai-model.ts:35
The fully qualified model resource name to use for generating images
(for example, publishers/google/models/imagen-3.0-generate-002).
normalizeModelName()
staticnormalizeModelName(modelName,backendType):string
Defined in: ai/lib/models/ai-model.ts:72
Internal
Normalizes the given model name to a fully qualified model resource name.
ArraySchema
Defined in: ai/lib/requests/schema-builder.ts:216
Schema class for "array" types.
The items param should refer to the type of item that can be a member
of the array.
Indexable
[key: string]: unknown
Allows user to add other schema properties that have not yet been officially added to the SDK.
Constructor
new ArraySchema(
schemaParams,items):ArraySchema
Defined in: ai/lib/requests/schema-builder.ts:217
description?
optionaldescription:string
Defined in: ai/lib/requests/schema-builder.ts:51
Optional. The description of the property.
example?
optionalexample:unknown
Defined in: ai/lib/requests/schema-builder.ts:61
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/requests/schema-builder.ts:49
Optional. The format of the property.
Supported formats:
- for NUMBER type: "float", "double"
- for INTEGER type: "int32", "int64"
- for STRING type: "email", "byte", etc
items
items:
TypedSchema
Defined in: ai/lib/requests/schema-builder.ts:219
Optional. The items of the property.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/requests/schema-builder.ts:57
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minItems?
optionalminItems:number
Defined in: ai/lib/requests/schema-builder.ts:55
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable
nullable:
boolean
Defined in: ai/lib/requests/schema-builder.ts:59
Optional. Whether the property is nullable. Defaults to false.
type
type:
SchemaType
Defined in: ai/lib/requests/schema-builder.ts:40
Optional. The type of the property. SchemaType.
toJSON()
toJSON():
SchemaRequest
Defined in: ai/lib/requests/schema-builder.ts:230
Internal
array()
staticarray(arrayParams):ArraySchema
Defined in: ai/lib/requests/schema-builder.ts:96
arrayParams
SchemaParams & object
boolean()
staticboolean(booleanParams?):BooleanSchema
Defined in: ai/lib/requests/schema-builder.ts:127
enumString()
staticenumString(stringParams):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:115
stringParams
SchemaParams & object
integer()
staticinteger(integerParams?):IntegerSchema
Defined in: ai/lib/requests/schema-builder.ts:119
number()
staticnumber(numberParams?):NumberSchema
Defined in: ai/lib/requests/schema-builder.ts:123
object()
staticobject(objectParams):ObjectSchema
Defined in: ai/lib/requests/schema-builder.ts:100
objectParams
SchemaParams & object
string()
staticstring(stringParams?):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:111
abstract Backend
Defined in: ai/lib/backend.ts:29
Abstract base class representing the configuration for an AI service backend. This class should not be instantiated directly. Use its subclasses; GoogleAIBackend for the Gemini Developer API (via Google AI), and VertexAIBackend for the Vertex AI Gemini API.
Constructor
protectednew Backend(type):Backend
Defined in: ai/lib/backend.ts:39
Protected constructor for use by subclasses.
backendType
readonlybackendType:BackendType
Defined in: ai/lib/backend.ts:33
Specifies the backend type.
_getModelPath()
abstract_getModelPath(project,model):string
Defined in: ai/lib/backend.ts:46
Internal
_getTemplatePath()
abstract_getTemplatePath(project,templateId):string
Defined in: ai/lib/backend.ts:51
Internal
BooleanSchema
Defined in: ai/lib/requests/schema-builder.ts:174
Schema class for "boolean" types.
Indexable
[key: string]: unknown
Allows user to add other schema properties that have not yet been officially added to the SDK.
Constructor
new BooleanSchema(
schemaParams?):BooleanSchema
Defined in: ai/lib/requests/schema-builder.ts:175
description?
optionaldescription:string
Defined in: ai/lib/requests/schema-builder.ts:51
Optional. The description of the property.
example?
optionalexample:unknown
Defined in: ai/lib/requests/schema-builder.ts:61
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/requests/schema-builder.ts:49
Optional. The format of the property.
Supported formats:
- for NUMBER type: "float", "double"
- for INTEGER type: "int32", "int64"
- for STRING type: "email", "byte", etc
items?
optionalitems:SchemaInterface
Defined in: ai/lib/requests/schema-builder.ts:53
Optional. The items of the property.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/requests/schema-builder.ts:57
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minItems?
optionalminItems:number
Defined in: ai/lib/requests/schema-builder.ts:55
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable
nullable:
boolean
Defined in: ai/lib/requests/schema-builder.ts:59
Optional. Whether the property is nullable. Defaults to false.
type
type:
SchemaType
Defined in: ai/lib/requests/schema-builder.ts:40
Optional. The type of the property. SchemaType.
toJSON()
toJSON():
SchemaRequest
Defined in: ai/lib/requests/schema-builder.ts:82
Internal
Defines how this Schema should be serialized as JSON. See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#tojson_behavior
array()
staticarray(arrayParams):ArraySchema
Defined in: ai/lib/requests/schema-builder.ts:96
arrayParams
SchemaParams & object
boolean()
staticboolean(booleanParams?):BooleanSchema
Defined in: ai/lib/requests/schema-builder.ts:127
enumString()
staticenumString(stringParams):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:115
stringParams
SchemaParams & object
integer()
staticinteger(integerParams?):IntegerSchema
Defined in: ai/lib/requests/schema-builder.ts:119
number()
staticnumber(numberParams?):NumberSchema
Defined in: ai/lib/requests/schema-builder.ts:123
object()
staticobject(objectParams):ObjectSchema
Defined in: ai/lib/requests/schema-builder.ts:100
objectParams
SchemaParams & object
string()
staticstring(stringParams?):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:111
ChatSession
Defined in: ai/lib/methods/chat-session.ts:46
ChatSession class that enables sending chat messages and stores history of sent and received messages so far.
Constructor
new ChatSession(
apiSettings,model,params?,requestOptions?):ChatSession
Defined in: ai/lib/methods/chat-session.ts:51
model
model:
string
Defined in: ai/lib/methods/chat-session.ts:53
params?
optionalparams:StartChatParams
Defined in: ai/lib/methods/chat-session.ts:54
requestOptions?
optionalrequestOptions:RequestOptions
Defined in: ai/lib/methods/chat-session.ts:55
getHistory()
getHistory():
Promise<Content[]>
Defined in: ai/lib/methods/chat-session.ts:69
Gets the chat history so far. Blocked prompts are not added to history. Neither blocked candidates nor the prompts that generated them are added to history.
Returns
Promise<Content[]>
sendMessage()
sendMessage(
request):Promise<GenerateContentResult>
Defined in: ai/lib/methods/chat-session.ts:78
Sends a chat message and receives a non-streaming GenerateContentResult
request
string | (string | Part)[]
Returns
Promise<GenerateContentResult>
sendMessageStream()
sendMessageStream(
request):Promise<GenerateContentStreamResult>
Defined in: ai/lib/methods/chat-session.ts:123
Sends a chat message and receives the response as a GenerateContentStreamResult containing an iterable stream and a response promise.
request
string | (string | Part)[]
Returns
Promise<GenerateContentStreamResult>
GenerativeModel
Defined in: ai/lib/models/generative-model.ts:45
Class for generative model APIs.
Constructor
new GenerativeModel(
ai,modelParams,requestOptions?):GenerativeModel
Defined in: ai/lib/models/generative-model.ts:53
_apiSettings
protected_apiSettings:ApiSettings
Defined in: ai/lib/models/ai-model.ts:40
Internal
generationConfig
generationConfig:
GenerationConfig
Defined in: ai/lib/models/generative-model.ts:46
model
readonlymodel:string
Defined in: ai/lib/models/ai-model.ts:35
The fully qualified model resource name to use for generating images
(for example, publishers/google/models/imagen-3.0-generate-002).
requestOptions?
optionalrequestOptions:RequestOptions
Defined in: ai/lib/models/generative-model.ts:48
safetySettings
safetySettings:
SafetySetting[]
Defined in: ai/lib/models/generative-model.ts:47
systemInstruction?
optionalsystemInstruction:Content
Defined in: ai/lib/models/generative-model.ts:51
toolConfig?
optionaltoolConfig:ToolConfig
Defined in: ai/lib/models/generative-model.ts:50
tools?
optionaltools:Tool[]
Defined in: ai/lib/models/generative-model.ts:49
countTokens()
countTokens(
request):Promise<CountTokensResponse>
Defined in: ai/lib/models/generative-model.ts:139
Counts the tokens in the provided request.
request
string | CountTokensRequest | (string | Part)[]
Returns
Promise<CountTokensResponse>
generateContent()
generateContent(
request):Promise<GenerateContentResult>
Defined in: ai/lib/models/generative-model.ts:67
Makes a single non-streaming call to the model and returns an object containing a single GenerateContentResponse.
request
string | GenerateContentRequest | (string | Part)[]
Returns
Promise<GenerateContentResult>
generateContentStream()
generateContentStream(
request):Promise<GenerateContentStreamResult>
Defined in: ai/lib/models/generative-model.ts:92
Makes a single streaming call to the model and returns an object containing an iterable stream that iterates over all chunks in the streaming response as well as a promise that returns the final aggregated response.
request
string | GenerateContentRequest | (string | Part)[]
Returns
Promise<GenerateContentStreamResult>
startChat()
startChat(
startChatParams?):ChatSession
Defined in: ai/lib/models/generative-model.ts:115
Gets a new ChatSession instance which can be used for multi-turn chats.
normalizeModelName()
staticnormalizeModelName(modelName,backendType):string
Defined in: ai/lib/models/ai-model.ts:72
Internal
Normalizes the given model name to a fully qualified model resource name.
GoogleAIBackend
Defined in: ai/lib/backend.ts:62
Configuration class for the Gemini Developer API.
Use this with AIOptions when initializing the AI service via getAI() to specify the Gemini Developer API as the backend.
Constructor
new GoogleAIBackend():
GoogleAIBackend
Defined in: ai/lib/backend.ts:66
Creates a configuration object for the Gemini Developer API backend.
backendType
readonlybackendType:BackendType
Defined in: ai/lib/backend.ts:33
Specifies the backend type.
_getModelPath()
_getModelPath(
project,model):string
Defined in: ai/lib/backend.ts:73
Internal
_getTemplatePath()
_getTemplatePath(
project,templateId):string
Defined in: ai/lib/backend.ts:80
Internal
ImagenImageFormat
Defined in: ai/lib/requests/imagen-image-format.ts:37
Beta
Defines the image format for images generated by Imagen.
Use this class to specify the desired format (JPEG or PNG) and compression quality for images generated by Imagen. This is typically included as part of ImagenModelParams.
Example
const imagenModelParams = {
// ... other ImagenModelParams
imageFormat: ImagenImageFormat.jpeg(75) // JPEG with a compression level of 75.
}
compressionQuality?
optionalcompressionQuality:number
Defined in: ai/lib/requests/imagen-image-format.ts:45
Beta
The level of compression (a number between 0 and 100).
mimeType
mimeType:
string
Defined in: ai/lib/requests/imagen-image-format.ts:41
Beta
The MIME type.
jpeg()
staticjpeg(compressionQuality?):ImagenImageFormat
Defined in: ai/lib/requests/imagen-image-format.ts:59
Beta
Creates an ImagenImageFormat for a JPEG image.
Returns
An ImagenImageFormat object for a JPEG image.
png()
staticpng():ImagenImageFormat
Defined in: ai/lib/requests/imagen-image-format.ts:75
Beta
Creates an ImagenImageFormat for a PNG image.
Returns
An ImagenImageFormat object for a PNG image.
ImagenModel
Defined in: ai/lib/models/imagen-model.ts:55
Beta
Class for Imagen model APIs.
This class provides methods for generating images using the Imagen model.
Example
const imagen = new ImagenModel(
ai,
{
model: 'imagen-3.0-generate-002'
}
);
const response = await imagen.generateImages('A photo of a cat');
if (response.images.length > 0) {
console.log(response.images[0].bytesBase64Encoded);
}
Constructor
new ImagenModel(
ai,modelParams,requestOptions?):ImagenModel
Defined in: ai/lib/models/imagen-model.ts:75
Beta
Constructs a new instance of the ImagenModel class.
_apiSettings
protected_apiSettings:ApiSettings
Defined in: ai/lib/models/ai-model.ts:40
Internal
generationConfig?
optionalgenerationConfig:ImagenGenerationConfig
Defined in: ai/lib/models/imagen-model.ts:59
Beta
The Imagen generation configuration.
model
readonlymodel:string
Defined in: ai/lib/models/ai-model.ts:35
Beta
The fully qualified model resource name to use for generating images
(for example, publishers/google/models/imagen-3.0-generate-002).
requestOptions?
optionalrequestOptions:RequestOptions
Defined in: ai/lib/models/imagen-model.ts:78
Beta
Additional options to use when making requests.
safetySettings?
optionalsafetySettings:ImagenSafetySettings
Defined in: ai/lib/models/imagen-model.ts:63
Beta
Safety settings for filtering inappropriate content.
generateImages()
generateImages(
prompt):Promise<ImagenGenerationResponse<ImagenInlineImage>>
Defined in: ai/lib/models/imagen-model.ts:104
Beta
Generates images using the Imagen model and returns them as base64-encoded strings.
Returns
Promise<ImagenGenerationResponse<ImagenInlineImage>>
A promise that resolves to an ImagenGenerationResponse object containing the generated images.
Remarks
If the prompt was not blocked, but one or more of the generated images were filtered, the
returned object will have a filteredReason property.
If all images are filtered, the images array will be empty.
generateImagesGCS()
generateImagesGCS(
prompt,gcsURI):Promise<ImagenGenerationResponse<ImagenGCSImage>>
Defined in: ai/lib/models/imagen-model.ts:141
Internal
Generates images to Cloud Storage for Firebase using the Imagen model.
This method is temporarily internal.
gcsURI
string
The URI of file stored in a Cloud Storage for Firebase bucket.
This should be a directory. For example, gs://my-bucket/my-directory/.
Returns
Promise<ImagenGenerationResponse<ImagenGCSImage>>
A promise that resolves to an ImagenGenerationResponse object containing the URLs of the generated images.
Remarks
If the prompt was not blocked, but one or more of the generated images were filtered, the
returned object will have a filteredReason property.
If all images are filtered, the images array will be empty.
normalizeModelName()
staticnormalizeModelName(modelName,backendType):string
Defined in: ai/lib/models/ai-model.ts:72
Internal
Normalizes the given model name to a fully qualified model resource name.
IntegerSchema
Defined in: ai/lib/requests/schema-builder.ts:148
Schema class for "integer" types.
Indexable
[key: string]: unknown
Allows user to add other schema properties that have not yet been officially added to the SDK.
Constructor
new IntegerSchema(
schemaParams?):IntegerSchema
Defined in: ai/lib/requests/schema-builder.ts:149
description?
optionaldescription:string
Defined in: ai/lib/requests/schema-builder.ts:51
Optional. The description of the property.
example?
optionalexample:unknown
Defined in: ai/lib/requests/schema-builder.ts:61
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/requests/schema-builder.ts:49
Optional. The format of the property.
Supported formats:
- for NUMBER type: "float", "double"
- for INTEGER type: "int32", "int64"
- for STRING type: "email", "byte", etc
items?
optionalitems:SchemaInterface
Defined in: ai/lib/requests/schema-builder.ts:53
Optional. The items of the property.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/requests/schema-builder.ts:57
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minItems?
optionalminItems:number
Defined in: ai/lib/requests/schema-builder.ts:55
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable
nullable:
boolean
Defined in: ai/lib/requests/schema-builder.ts:59
Optional. Whether the property is nullable. Defaults to false.
type
type:
SchemaType
Defined in: ai/lib/requests/schema-builder.ts:40
Optional. The type of the property. SchemaType.
toJSON()
toJSON():
SchemaRequest
Defined in: ai/lib/requests/schema-builder.ts:82
Internal
Defines how this Schema should be serialized as JSON. See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#tojson_behavior
array()
staticarray(arrayParams):ArraySchema
Defined in: ai/lib/requests/schema-builder.ts:96
arrayParams
SchemaParams & object
boolean()
staticboolean(booleanParams?):BooleanSchema
Defined in: ai/lib/requests/schema-builder.ts:127
enumString()
staticenumString(stringParams):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:115
stringParams
SchemaParams & object
integer()
staticinteger(integerParams?):IntegerSchema
Defined in: ai/lib/requests/schema-builder.ts:119
number()
staticnumber(numberParams?):NumberSchema
Defined in: ai/lib/requests/schema-builder.ts:123
object()
staticobject(objectParams):ObjectSchema
Defined in: ai/lib/requests/schema-builder.ts:100
objectParams
SchemaParams & object
string()
staticstring(stringParams?):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:111
LiveGenerativeModel
Defined in: ai/lib/models/live-generative-model.ts:44
Beta
Class for Live generative model APIs. The Live API enables low-latency, two-way multimodal interactions with Gemini.
This class should only be instantiated with getLiveGenerativeModel.
Constructor
new LiveGenerativeModel(
ai,modelParams,_webSocketHandler):LiveGenerativeModel
Defined in: ai/lib/models/live-generative-model.ts:53
Internal
_apiSettings
protected_apiSettings:ApiSettings
Defined in: ai/lib/models/ai-model.ts:40
Internal
generationConfig
generationConfig:
LiveGenerationConfig
Defined in: ai/lib/models/live-generative-model.ts:45
Beta
model
readonlymodel:string
Defined in: ai/lib/models/ai-model.ts:35
Beta
The fully qualified model resource name to use for generating images
(for example, publishers/google/models/imagen-3.0-generate-002).
systemInstruction?
optionalsystemInstruction:Content
Defined in: ai/lib/models/live-generative-model.ts:48
Beta
toolConfig?
optionaltoolConfig:ToolConfig
Defined in: ai/lib/models/live-generative-model.ts:47
Beta
tools?
optionaltools:Tool[]
Defined in: ai/lib/models/live-generative-model.ts:46
Beta
connect()
connect():
Promise<LiveSession>
Defined in: ai/lib/models/live-generative-model.ts:76
Beta
Starts a LiveSession.
Returns
Promise<LiveSession>
A LiveSession.
normalizeModelName()
staticnormalizeModelName(modelName,backendType):string
Defined in: ai/lib/models/ai-model.ts:72
Internal
Normalizes the given model name to a fully qualified model resource name.
LiveSession
Defined in: ai/lib/methods/live-session.ts:46
Beta
Represents an active, real-time, bidirectional conversation with the model.
This class should only be instantiated by calling LiveGenerativeModel.connect.
Constructor
new LiveSession(
webSocketHandler,serverMessages):LiveSession
Defined in: ai/lib/methods/live-session.ts:63
Internal
inConversation
inConversation:
boolean=false
Defined in: ai/lib/methods/live-session.ts:58
Beta
Indicates whether this Live session is being controlled by an AudioConversationController.
isClosed
isClosed:
boolean=false
Defined in: ai/lib/methods/live-session.ts:52
Beta
Indicates whether this Live session is closed.
close()
close():
Promise<void>
Defined in: ai/lib/methods/live-session.ts:272
Beta
Closes this session. All methods on this session will throw an error once this resolves.
receive()
receive():
AsyncGenerator<LiveServerContent|LiveServerToolCall|LiveServerToolCallCancellation>
Defined in: ai/lib/methods/live-session.ts:225
Beta
Yields messages received from the server. This can only be used by one consumer at a time.
Returns
AsyncGenerator<LiveServerContent | LiveServerToolCall | LiveServerToolCallCancellation>
An AsyncGenerator that yields server messages as they arrive.
send()
send(
request,turnComplete?):Promise<void>
Defined in: ai/lib/methods/live-session.ts:77
Beta
Sends content to the server.
request
The message to send to the model.
string | (string | Part)[]
sendAudioRealtime()
sendAudioRealtime(
blob):Promise<void>
Defined in: ai/lib/methods/live-session.ts:143
Beta
Sends audio data to the server in realtime.
Remarks
The server requires that the audio data is base64-encoded 16-bit PCM at 16kHz little-endian.
Example
// const pcmData = ... base64-encoded 16-bit PCM at 16kHz little-endian.
const blob = { mimeType: "audio/pcm", data: pcmData };
liveSession.sendAudioRealtime(blob);
sendFunctionResponses()
sendFunctionResponses(
functionResponses):Promise<void>
Defined in: ai/lib/methods/live-session.ts:200
Beta
Sends function responses to the server.
sendMediaChunks()
sendMediaChunks(
mediaChunks):Promise<void>
Defined in: ai/lib/methods/live-session.ts:289
Beta
Sends realtime input to the server.
sendMediaStream()
sendMediaStream(
mediaChunkStream):Promise<void>
Defined in: ai/lib/methods/live-session.ts:317
Beta
mediaChunkStream
ReadableStream<GenerativeContentBlob>
The stream of GenerativeContentBlob to send.
Deprecated
Use sendTextRealtime(), sendAudioRealtime(), and sendVideoRealtime() instead.
Sends a stream of GenerativeContentBlob.
sendTextRealtime()
sendTextRealtime(
text):Promise<void>
Defined in: ai/lib/methods/live-session.ts:109
Beta
Sends text to the server in realtime.
sendVideoRealtime()
sendVideoRealtime(
blob):Promise<void>
Defined in: ai/lib/methods/live-session.ts:176
Beta
Sends video data to the server in realtime.
Remarks
The server requires that the video is sent as individual video frames at 1 FPS. It
is recommended to set mimeType to image/jpeg.
Example
// const videoFrame = ... base64-encoded JPEG data
const blob = { mimeType: "image/jpeg", data: videoFrame };
liveSession.sendVideoRealtime(blob);
NumberSchema
Defined in: ai/lib/requests/schema-builder.ts:161
Schema class for "number" types.
Indexable
[key: string]: unknown
Allows user to add other schema properties that have not yet been officially added to the SDK.
Constructor
new NumberSchema(
schemaParams?):NumberSchema
Defined in: ai/lib/requests/schema-builder.ts:162
description?
optionaldescription:string
Defined in: ai/lib/requests/schema-builder.ts:51
Optional. The description of the property.
example?
optionalexample:unknown
Defined in: ai/lib/requests/schema-builder.ts:61
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/requests/schema-builder.ts:49
Optional. The format of the property.
Supported formats:
- for NUMBER type: "float", "double"
- for INTEGER type: "int32", "int64"
- for STRING type: "email", "byte", etc
items?
optionalitems:SchemaInterface
Defined in: ai/lib/requests/schema-builder.ts:53
Optional. The items of the property.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/requests/schema-builder.ts:57
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minItems?
optionalminItems:number
Defined in: ai/lib/requests/schema-builder.ts:55
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable
nullable:
boolean
Defined in: ai/lib/requests/schema-builder.ts:59
Optional. Whether the property is nullable. Defaults to false.
type
type:
SchemaType
Defined in: ai/lib/requests/schema-builder.ts:40
Optional. The type of the property. SchemaType.
toJSON()
toJSON():
SchemaRequest
Defined in: ai/lib/requests/schema-builder.ts:82
Internal
Defines how this Schema should be serialized as JSON. See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#tojson_behavior
array()
staticarray(arrayParams):ArraySchema
Defined in: ai/lib/requests/schema-builder.ts:96
arrayParams
SchemaParams & object
boolean()
staticboolean(booleanParams?):BooleanSchema
Defined in: ai/lib/requests/schema-builder.ts:127
enumString()
staticenumString(stringParams):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:115
stringParams
SchemaParams & object
integer()
staticinteger(integerParams?):IntegerSchema
Defined in: ai/lib/requests/schema-builder.ts:119
number()
staticnumber(numberParams?):NumberSchema
Defined in: ai/lib/requests/schema-builder.ts:123
object()
staticobject(objectParams):ObjectSchema
Defined in: ai/lib/requests/schema-builder.ts:100
objectParams
SchemaParams & object
string()
staticstring(stringParams?):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:111
ObjectSchema
Defined in: ai/lib/requests/schema-builder.ts:242
Schema class for "object" types.
The properties param must be a map of Schema objects.
Indexable
[key: string]: unknown
Allows user to add other schema properties that have not yet been officially added to the SDK.
Constructor
new ObjectSchema(
schemaParams,properties,optionalProperties?):ObjectSchema
Defined in: ai/lib/requests/schema-builder.ts:243
description?
optionaldescription:string
Defined in: ai/lib/requests/schema-builder.ts:51
Optional. The description of the property.
example?
optionalexample:unknown
Defined in: ai/lib/requests/schema-builder.ts:61
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/requests/schema-builder.ts:49
Optional. The format of the property.
Supported formats:
- for NUMBER type: "float", "double"
- for INTEGER type: "int32", "int64"
- for STRING type: "email", "byte", etc
items?
optionalitems:SchemaInterface
Defined in: ai/lib/requests/schema-builder.ts:53
Optional. The items of the property.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/requests/schema-builder.ts:57
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minItems?
optionalminItems:number
Defined in: ai/lib/requests/schema-builder.ts:55
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable
nullable:
boolean
Defined in: ai/lib/requests/schema-builder.ts:59
Optional. Whether the property is nullable. Defaults to false.
optionalProperties
optionalProperties:
string[] =[]
Defined in: ai/lib/requests/schema-builder.ts:248
properties
properties:
object
Defined in: ai/lib/requests/schema-builder.ts:245
Index Signature
[k: string]: TypedSchema
type
type:
SchemaType
Defined in: ai/lib/requests/schema-builder.ts:40
Optional. The type of the property. SchemaType.
toJSON()
toJSON():
SchemaRequest
Defined in: ai/lib/requests/schema-builder.ts:259
Internal
array()
staticarray(arrayParams):ArraySchema
Defined in: ai/lib/requests/schema-builder.ts:96
arrayParams
SchemaParams & object
boolean()
staticboolean(booleanParams?):BooleanSchema
Defined in: ai/lib/requests/schema-builder.ts:127
enumString()
staticenumString(stringParams):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:115
stringParams
SchemaParams & object
integer()
staticinteger(integerParams?):IntegerSchema
Defined in: ai/lib/requests/schema-builder.ts:119
number()
staticnumber(numberParams?):NumberSchema
Defined in: ai/lib/requests/schema-builder.ts:123
object()
staticobject(objectParams):ObjectSchema
Defined in: ai/lib/requests/schema-builder.ts:100
objectParams
SchemaParams & object
string()
staticstring(stringParams?):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:111
abstract Schema
Defined in: ai/lib/requests/schema-builder.ts:35
Parent class encompassing all Schema types, with static methods that
allow building specific Schema types. This class can be converted with
JSON.stringify() into a JSON string accepted by Vertex AI REST endpoints.
(This string conversion is automatically done when calling SDK methods.)
Indexable
[key: string]: unknown
Allows user to add other schema properties that have not yet been officially added to the SDK.
Constructor
new Schema(
schemaParams):Schema
Defined in: ai/lib/requests/schema-builder.ts:68
description?
optionaldescription:string
Defined in: ai/lib/requests/schema-builder.ts:51
Optional. The description of the property.
example?
optionalexample:unknown
Defined in: ai/lib/requests/schema-builder.ts:61
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/requests/schema-builder.ts:49
Optional. The format of the property.
Supported formats:
- for NUMBER type: "float", "double"
- for INTEGER type: "int32", "int64"
- for STRING type: "email", "byte", etc
items?
optionalitems:SchemaInterface
Defined in: ai/lib/requests/schema-builder.ts:53
Optional. The items of the property.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/requests/schema-builder.ts:57
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minItems?
optionalminItems:number
Defined in: ai/lib/requests/schema-builder.ts:55
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable
nullable:
boolean
Defined in: ai/lib/requests/schema-builder.ts:59
Optional. Whether the property is nullable. Defaults to false.
type
type:
SchemaType
Defined in: ai/lib/requests/schema-builder.ts:40
Optional. The type of the property. SchemaType.
toJSON()
toJSON():
SchemaRequest
Defined in: ai/lib/requests/schema-builder.ts:82
Internal
Defines how this Schema should be serialized as JSON. See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#tojson_behavior
array()
staticarray(arrayParams):ArraySchema
Defined in: ai/lib/requests/schema-builder.ts:96
arrayParams
SchemaParams & object
boolean()
staticboolean(booleanParams?):BooleanSchema
Defined in: ai/lib/requests/schema-builder.ts:127
enumString()
staticenumString(stringParams):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:115
stringParams
SchemaParams & object
integer()
staticinteger(integerParams?):IntegerSchema
Defined in: ai/lib/requests/schema-builder.ts:119
number()
staticnumber(numberParams?):NumberSchema
Defined in: ai/lib/requests/schema-builder.ts:123
object()
staticobject(objectParams):ObjectSchema
Defined in: ai/lib/requests/schema-builder.ts:100
objectParams
SchemaParams & object
string()
staticstring(stringParams?):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:111
StringSchema
Defined in: ai/lib/requests/schema-builder.ts:188
Schema class for "string" types. Can be used with or without enum values.
Indexable
[key: string]: unknown
Allows user to add other schema properties that have not yet been officially added to the SDK.
Constructor
new StringSchema(
schemaParams?,enumValues?):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:190
description?
optionaldescription:string
Defined in: ai/lib/requests/schema-builder.ts:51
Optional. The description of the property.
enum?
optionalenum:string[]
Defined in: ai/lib/requests/schema-builder.ts:189
example?
optionalexample:unknown
Defined in: ai/lib/requests/schema-builder.ts:61
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/requests/schema-builder.ts:49
Optional. The format of the property.
Supported formats:
- for NUMBER type: "float", "double"
- for INTEGER type: "int32", "int64"
- for STRING type: "email", "byte", etc
items?
optionalitems:SchemaInterface
Defined in: ai/lib/requests/schema-builder.ts:53
Optional. The items of the property.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/requests/schema-builder.ts:57
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minItems?
optionalminItems:number
Defined in: ai/lib/requests/schema-builder.ts:55
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable
nullable:
boolean
Defined in: ai/lib/requests/schema-builder.ts:59
Optional. Whether the property is nullable. Defaults to false.
type
type:
SchemaType
Defined in: ai/lib/requests/schema-builder.ts:40
Optional. The type of the property. SchemaType.
toJSON()
toJSON():
SchemaRequest
Defined in: ai/lib/requests/schema-builder.ts:201
Internal
array()
staticarray(arrayParams):ArraySchema
Defined in: ai/lib/requests/schema-builder.ts:96
arrayParams
SchemaParams & object
boolean()
staticboolean(booleanParams?):BooleanSchema
Defined in: ai/lib/requests/schema-builder.ts:127
enumString()
staticenumString(stringParams):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:115
stringParams
SchemaParams & object
integer()
staticinteger(integerParams?):IntegerSchema
Defined in: ai/lib/requests/schema-builder.ts:119
number()
staticnumber(numberParams?):NumberSchema
Defined in: ai/lib/requests/schema-builder.ts:123
object()
staticobject(objectParams):ObjectSchema
Defined in: ai/lib/requests/schema-builder.ts:100
objectParams
SchemaParams & object
string()
staticstring(stringParams?):StringSchema
Defined in: ai/lib/requests/schema-builder.ts:111
TemplateGenerativeModel
Defined in: ai/lib/models/template-generative-model.ts:34
Beta
GenerativeModel APIs that execute on a server-side template.
This class should only be instantiated with getTemplateGenerativeModel.
_apiSettings
_apiSettings:
ApiSettings
Defined in: ai/lib/models/template-generative-model.ts:38
Internal
requestOptions?
optionalrequestOptions:RequestOptions
Defined in: ai/lib/models/template-generative-model.ts:43
Beta
Additional options to use when making requests.
generateContent()
generateContent(
templateId,templateVariables):Promise<GenerateContentResult>
Defined in: ai/lib/models/template-generative-model.ts:63
Beta
Makes a single non-streaming call to the model and returns an object containing a single GenerateContentResponse.
Returns
Promise<GenerateContentResult>
generateContentStream()
generateContentStream(
templateId,templateVariables):Promise<GenerateContentStreamResult>
Defined in: ai/lib/models/template-generative-model.ts:87
Beta
Makes a single streaming call to the model and returns an object containing an iterable stream that iterates over all chunks in the streaming response as well as a promise that returns the final aggregated response.
Returns
Promise<GenerateContentStreamResult>
TemplateImagenModel
Defined in: ai/lib/models/template-imagen-model.ts:32
Beta
Class for Imagen model APIs that execute on a server-side template.
This class should only be instantiated with getTemplateImagenModel.
_apiSettings
_apiSettings:
ApiSettings
Defined in: ai/lib/models/template-imagen-model.ts:36
Internal
requestOptions?
optionalrequestOptions:RequestOptions
Defined in: ai/lib/models/template-imagen-model.ts:41
Beta
Additional options to use when making requests.
generateImages()
generateImages(
templateId,templateVariables):Promise<ImagenGenerationResponse<ImagenInlineImage>>
Defined in: ai/lib/models/template-imagen-model.ts:61
Beta
Makes a single call to the model and returns an object containing a single ImagenGenerationResponse.
Returns
Promise<ImagenGenerationResponse<ImagenInlineImage>>
VertexAIBackend
Defined in: ai/lib/backend.ts:93
Configuration class for the Vertex AI Gemini API.
Use this with AIOptions when initializing the AI service via getAI() to specify the Vertex AI Gemini API as the backend.
Constructor
new VertexAIBackend(
location?):VertexAIBackend
Defined in: ai/lib/backend.ts:108
Creates a configuration object for the Vertex AI backend.
location?
string = DEFAULT_LOCATION
The region identifier, defaulting to us-central1;
see Vertex AI locations
for a list of supported locations.
backendType
readonlybackendType:BackendType
Defined in: ai/lib/backend.ts:33
Specifies the backend type.
location
readonlylocation:string
Defined in: ai/lib/backend.ts:99
The region identifier. See Vertex AI locations for a list of supported locations.
_getModelPath()
_getModelPath(
project,model):string
Defined in: ai/lib/backend.ts:120
Internal
_getTemplatePath()
_getTemplatePath(
project,templateId):string
Defined in: ai/lib/backend.ts:127
Internal
AI
Defined in: ai/lib/public-types.ts:88
An instance of the Firebase AI SDK.
Do not create this instance directly. Instead, use getAI().
app
app:
FirebaseApp
Defined in: ai/lib/public-types.ts:92
The @firebase/app#FirebaseApp this AI instance is associated with.
appCheck?
optionalappCheck:AppCheck|null
Defined in: ai/lib/public-types.ts:93
auth?
optionalauth:Module|null
Defined in: ai/lib/public-types.ts:94
backend
backend:
Backend
Defined in: ai/lib/public-types.ts:100
A Backend instance that specifies the configuration for the target backend, either the Gemini Developer API (using GoogleAIBackend) or the Vertex AI Gemini API (using VertexAIBackend).
location
location:
string
Defined in: ai/lib/public-types.ts:110
Deprecated
use AI.backend.location instead.
The location configured for this AI service instance, relevant for Vertex AI backends.
options?
optionaloptions:AIOptions
Defined in: ai/lib/public-types.ts:104
Options applied to this AI instance.
AIOptions
Defined in: ai/lib/public-types.ts:33
Options for initializing the AI service using getAI(). This allows specifying which backend to use (Vertex AI Gemini API or Gemini Developer API) and configuring its specific options (like location for Vertex AI).
appCheck?
optionalappCheck:AppCheck|null
Defined in: ai/lib/public-types.ts:43
auth?
optionalauth:Module|null
Defined in: ai/lib/public-types.ts:44
backend?
optionalbackend:Backend
Defined in: ai/lib/public-types.ts:38
The backend configuration to use for the AI service instance. Defaults to the Gemini Developer API backend (GoogleAIBackend).
useLimitedUseAppCheckTokens?
optionaluseLimitedUseAppCheckTokens:boolean
Defined in: ai/lib/public-types.ts:42
Whether to use App Check limited use tokens. Defaults to false.
AudioTranscriptionConfig
Defined in: ai/lib/types/requests.ts:434
Beta
Configuration for audio transcription in Live sessions.
BaseParams
Defined in: ai/lib/types/requests.ts:33
Base parameters for a number of methods.
generationConfig?
optionalgenerationConfig:GenerationConfig
Defined in: ai/lib/types/requests.ts:35
safetySettings?
optionalsafetySettings:SafetySetting[]
Defined in: ai/lib/types/requests.ts:34
Citation
Defined in: ai/lib/types/responses.ts:178
A single citation.
endIndex?
optionalendIndex:number
Defined in: ai/lib/types/responses.ts:180
license?
optionallicense:string
Defined in: ai/lib/types/responses.ts:182
publicationDate?
optionalpublicationDate:Date
Defined in: ai/lib/types/responses.ts:194
The publication date of the cited source, if available.
This property is only supported in the Vertex AI Gemini API (VertexAIBackend).
startIndex?
optionalstartIndex:number
Defined in: ai/lib/types/responses.ts:179
title?
optionaltitle:string
Defined in: ai/lib/types/responses.ts:188
The title of the cited source, if available.
This property is only supported in the Vertex AI Gemini API (VertexAIBackend).
uri?
optionaluri:string
Defined in: ai/lib/types/responses.ts:181
CitationMetadata
Defined in: ai/lib/types/responses.ts:170
Citation metadata that may be found on a GenerateContentCandidate.
citations
citations:
Citation[]
Defined in: ai/lib/types/responses.ts:171
Content
Defined in: ai/lib/types/content.ts:24
Content type for both prompts and response candidates.
parts
parts:
Part[]
Defined in: ai/lib/types/content.ts:26
role
role:
"function"|"user"|"model"|"system"
Defined in: ai/lib/types/content.ts:25
CountTokensRequest
Defined in: ai/lib/types/requests.ts:218
Params for calling GenerativeModel.countTokens
contents
contents:
Content[]
Defined in: ai/lib/types/requests.ts:219
generationConfig?
optionalgenerationConfig:GenerationConfig
Defined in: ai/lib/types/requests.ts:231
Configuration options that control how the model generates a response.
systemInstruction?
Defined in: ai/lib/types/requests.ts:223
Instructions that direct the model to behave a certain way.
tools?
optionaltools:Tool[]
Defined in: ai/lib/types/requests.ts:227
Tool configuration.
CountTokensResponse
Defined in: ai/lib/types/responses.ts:395
Response from calling GenerativeModel.countTokens.
promptTokensDetails?
optionalpromptTokensDetails:ModalityTokenCount[]
Defined in: ai/lib/types/responses.ts:413
The breakdown, by modality, of how many tokens are consumed by the prompt.
totalBillableCharacters?
optionaltotalBillableCharacters:number
Defined in: ai/lib/types/responses.ts:409
Deprecated
Use totalTokens instead. This property is undefined when using models greater than gemini-1.5-*.
The total number of billable characters counted across all instances from the request.
This property is only supported when using the Vertex AI Gemini API (VertexAIBackend). When using the Gemini Developer API (GoogleAIBackend), this property is not supported and will default to 0.
totalTokens
totalTokens:
number
Defined in: ai/lib/types/responses.ts:399
The total number of tokens counted across all instances from the request.
CustomErrorData
Defined in: ai/lib/types/error.ts:46
Details object that contains data originating from a bad HTTP response.
errorDetails?
optionalerrorDetails:ErrorDetails[]
Defined in: ai/lib/types/error.ts:57
Optional additional details about the error.
response?
optionalresponse:GenerateContentResponse
Defined in: ai/lib/types/error.ts:54
Response from a GenerateContentRequest
status?
optionalstatus:number
Defined in: ai/lib/types/error.ts:48
HTTP status code of the error response.
statusText?
optionalstatusText:string
Defined in: ai/lib/types/error.ts:51
HTTP status text of the error response.
Date
Defined in: ai/lib/types/responses.ts:354
Protobuf google.type.Date
day
day:
number
Defined in: ai/lib/types/responses.ts:357
month
month:
number
Defined in: ai/lib/types/responses.ts:356
year
year:
number
Defined in: ai/lib/types/responses.ts:355
EnhancedGenerateContentResponse
Defined in: ai/lib/types/responses.ts:55
Response object wrapped with helper methods.
candidates?
optionalcandidates:GenerateContentCandidate[]
Defined in: ai/lib/types/responses.ts:102
functionCalls()
functionCalls: () =>
FunctionCall[] |undefined
Defined in: ai/lib/types/responses.ts:76
Aggregates and returns every FunctionCall from the first candidate of GenerateContentResponse.
Returns
FunctionCall[] | undefined
inlineDataParts()
inlineDataParts: () =>
InlineDataPart[] |undefined
Defined in: ai/lib/types/responses.ts:69
Aggregates and returns all InlineDataParts from the GenerateContentResponse's first candidate.
Returns
InlineDataPart[] | undefined
An array of InlineDataParts containing data from the response, if available.
promptFeedback?
optionalpromptFeedback:PromptFeedback
Defined in: ai/lib/types/responses.ts:103
text()
text: () =>
string
Defined in: ai/lib/types/responses.ts:60
Returns the text string from the response, if available. Throws if the prompt or candidate was blocked.
thoughtSummary()
thoughtSummary: () =>
string|undefined
Defined in: ai/lib/types/responses.ts:91
Aggregates and returns every TextPart with their thought property set
to true from the first candidate of GenerateContentResponse.
Remarks
Thought summaries provide a brief overview of the model's internal thinking process, offering insight into how it arrived at the final answer. This can be useful for debugging, understanding the model's reasoning, and verifying its accuracy.
Thoughts will only be included if ThinkingConfig.includeThoughts is
set to true.
usageMetadata?
optionalusageMetadata:UsageMetadata
Defined in: ai/lib/types/responses.ts:104
ErrorDetails
Defined in: ai/lib/types/error.ts:25
Details object that may be included in an error response.
@type?
optional@type:string
Defined in: ai/lib/types/error.ts:26
domain?
optionaldomain:string
Defined in: ai/lib/types/error.ts:32
The domain where the error occurred.
metadata?
optionalmetadata:Record<string,unknown>
Defined in: ai/lib/types/error.ts:35
Additional metadata about the error.
reason?
optionalreason:string
Defined in: ai/lib/types/error.ts:29
The reason for the error.
FileData
Defined in: ai/lib/types/content.ts:201
Data pointing to a file uploaded on Google Cloud Storage.
fileUri
fileUri:
string
Defined in: ai/lib/types/content.ts:203
mimeType
mimeType:
string
Defined in: ai/lib/types/content.ts:202
FileDataPart
Defined in: ai/lib/types/content.ts:130
Content part interface if the part represents FileData
fileData
fileData:
FileData
Defined in: ai/lib/types/content.ts:135
functionCall?
optionalfunctionCall:undefined
Defined in: ai/lib/types/content.ts:133
functionResponse?
optionalfunctionResponse:undefined
Defined in: ai/lib/types/content.ts:134
inlineData?
optionalinlineData:undefined
Defined in: ai/lib/types/content.ts:132
text?
optionaltext:undefined
Defined in: ai/lib/types/content.ts:131
thought?
optionalthought:boolean
Defined in: ai/lib/types/content.ts:136
thoughtSignature?
optionalthoughtSignature:undefined
Defined in: ai/lib/types/content.ts:140
Internal
FunctionCall
Defined in: ai/lib/types/content.ts:149
A predicted FunctionCall returned from the model that contains a string representing the FunctionDeclaration.name and a structured JSON object containing the parameters and their values.
args
args:
object
Defined in: ai/lib/types/content.ts:160
id?
optionalid:string
Defined in: ai/lib/types/content.ts:158
The id of the function call. This must be sent back in the associated FunctionResponse.
Remarks
This property is only supported in the Gemini Developer API (GoogleAIBackend).
When using the Gemini Developer API (GoogleAIBackend), this property will be
undefined.
name
name:
string
Defined in: ai/lib/types/content.ts:159
FunctionCallingConfig
Defined in: ai/lib/types/requests.ts:350
allowedFunctionNames?
optionalallowedFunctionNames:string[]
Defined in: ai/lib/types/requests.ts:352
mode?
optionalmode:FunctionCallingMode
Defined in: ai/lib/types/requests.ts:351
FunctionCallPart
Defined in: ai/lib/types/content.ts:98
Content part interface if the part represents a FunctionCall.
functionCall
functionCall:
FunctionCall
Defined in: ai/lib/types/content.ts:101
functionResponse?
optionalfunctionResponse:undefined
Defined in: ai/lib/types/content.ts:102
inlineData?
optionalinlineData:undefined
Defined in: ai/lib/types/content.ts:100
text?
optionaltext:undefined
Defined in: ai/lib/types/content.ts:99
thought?
optionalthought:boolean
Defined in: ai/lib/types/content.ts:103
thoughtSignature?
optionalthoughtSignature:undefined
Defined in: ai/lib/types/content.ts:107
Internal
FunctionDeclaration
Defined in: ai/lib/types/requests.ts:264
Structured representation of a function declaration as defined by the
OpenAPI 3.0 specification.
Included
in this declaration are the function name and parameters. This
FunctionDeclaration is a representation of a block of code that can be used
as a Tool by the model and executed by the client.
description
description:
string
Defined in: ai/lib/types/requests.ts:275
Description and purpose of the function. Model uses it to decide how and whether to call the function.
name
name:
string
Defined in: ai/lib/types/requests.ts:270
The name of the function to call. Must start with a letter or an underscore. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a max length of 64.
parameters?
optionalparameters:ObjectSchemaInterface
Defined in: ai/lib/types/requests.ts:281
Optional. Describes the parameters to this function in JSON Schema Object format. Reflects the Open API 3.03 Parameter Object. Parameter names are case-sensitive. For a function with no parameters, this can be left unset.
FunctionDeclarationsTool
Defined in: ai/lib/types/requests.ts:325
A FunctionDeclarationsTool is a piece of code that enables the system to
interact with external systems to perform an action, or set of actions,
outside of knowledge and scope of the model.
functionDeclarations?
optionalfunctionDeclarations:FunctionDeclaration[]
Defined in: ai/lib/types/requests.ts:336
Optional. One or more function declarations to be passed to the model along with the current user query. Model may decide to call a subset of these functions by populating FunctionCall in the response. User should provide a FunctionResponse for each function call in the next turn. Based on the function responses, the model will generate the final response back to the user. Maximum 64 function declarations can be provided.
FunctionResponse
Defined in: ai/lib/types/content.ts:172
The result output from a FunctionCall that contains a string representing the FunctionDeclaration.name and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a FunctionCall made based on model prediction.
id?
optionalid:string
Defined in: ai/lib/types/content.ts:180
The id of the FunctionCall.
Remarks
This property is only supported in the Gemini Developer API (GoogleAIBackend).
When using the Gemini Developer API (GoogleAIBackend), this property will be
undefined.
name
name:
string
Defined in: ai/lib/types/content.ts:181
response
response:
object
Defined in: ai/lib/types/content.ts:182
FunctionResponsePart
Defined in: ai/lib/types/content.ts:114
Content part interface if the part represents FunctionResponse.
functionCall?
optionalfunctionCall:undefined
Defined in: ai/lib/types/content.ts:117
functionResponse
functionResponse:
FunctionResponse
Defined in: ai/lib/types/content.ts:118
inlineData?
optionalinlineData:undefined
Defined in: ai/lib/types/content.ts:116
text?
optionaltext:undefined
Defined in: ai/lib/types/content.ts:115
thought?
optionalthought:boolean
Defined in: ai/lib/types/content.ts:119
thoughtSignature?
optionalthoughtSignature:undefined
Defined in: ai/lib/types/content.ts:123
Internal
GenerateContentCandidate
Defined in: ai/lib/types/responses.ts:156
A candidate returned as part of a GenerateContentResponse.
citationMetadata?
optionalcitationMetadata:CitationMetadata
Defined in: ai/lib/types/responses.ts:162
content
content:
Content
Defined in: ai/lib/types/responses.ts:158
finishMessage?
optionalfinishMessage:string
Defined in: ai/lib/types/responses.ts:160
finishReason?
optionalfinishReason:FinishReason
Defined in: ai/lib/types/responses.ts:159
groundingMetadata?
optionalgroundingMetadata:GroundingMetadata
Defined in: ai/lib/types/responses.ts:163
index
index:
number
Defined in: ai/lib/types/responses.ts:157
safetyRatings?
optionalsafetyRatings:SafetyRating[]
Defined in: ai/lib/types/responses.ts:161
GenerateContentRequest
Defined in: ai/lib/types/requests.ts:65
Request sent through GenerativeModel.generateContent
contents
contents:
Content[]
Defined in: ai/lib/types/requests.ts:66
generationConfig?
optionalgenerationConfig:GenerationConfig
Defined in: ai/lib/types/requests.ts:35
safetySettings?
optionalsafetySettings:SafetySetting[]
Defined in: ai/lib/types/requests.ts:34
systemInstruction?
Defined in: ai/lib/types/requests.ts:69
toolConfig?
optionaltoolConfig:ToolConfig
Defined in: ai/lib/types/requests.ts:68
tools?
optionaltools:Tool[]
Defined in: ai/lib/types/requests.ts:67
GenerateContentResponse
Defined in: ai/lib/types/responses.ts:101
Individual response from GenerativeModel.generateContent and
GenerativeModel.generateContentStream.
generateContentStream() will return one in each chunk until
the stream is done.
candidates?
optionalcandidates:GenerateContentCandidate[]
Defined in: ai/lib/types/responses.ts:102
promptFeedback?
optionalpromptFeedback:PromptFeedback
Defined in: ai/lib/types/responses.ts:103
usageMetadata?
optionalusageMetadata:UsageMetadata
Defined in: ai/lib/types/responses.ts:104
GenerateContentResult
Defined in: ai/lib/types/responses.ts:33
Result object returned from GenerativeModel.generateContent call.
response
response:
EnhancedGenerateContentResponse
Defined in: ai/lib/types/responses.ts:34
GenerateContentStreamResult
Defined in: ai/lib/types/responses.ts:45
Result object returned from GenerativeModel.generateContentStream call.
Iterate over stream to get chunks as they come in and/or
use the response promise to get the aggregated response when
the stream is done.
response
response:
Promise<EnhancedGenerateContentResponse>
Defined in: ai/lib/types/responses.ts:47
stream
stream:
AsyncGenerator<EnhancedGenerateContentResponse>
Defined in: ai/lib/types/responses.ts:46
GenerationConfig
Defined in: ai/lib/types/requests.ts:93
Config options for content-related requests
candidateCount?
optionalcandidateCount:number
Defined in: ai/lib/types/requests.ts:94
frequencyPenalty?
optionalfrequencyPenalty:number
Defined in: ai/lib/types/requests.ts:101
maxOutputTokens?
optionalmaxOutputTokens:number
Defined in: ai/lib/types/requests.ts:96
presencePenalty?
optionalpresencePenalty:number
Defined in: ai/lib/types/requests.ts:100
responseMimeType?
optionalresponseMimeType:string
Defined in: ai/lib/types/requests.ts:108
Output response MIME type of the generated candidate text.
Supported MIME types are text/plain (default, text output),
application/json (JSON response in the candidates), and
text/x.enum.
responseModalities?
optionalresponseModalities:ResponseModality[]
Defined in: ai/lib/types/requests.ts:127
Beta
Generation modalities to be returned in generation responses.
Remarks
- Multimodal response generation is only supported by some Gemini models and versions; see model versions.
- Only image generation (
ResponseModality.IMAGE) is supported.
responseSchema?
optionalresponseSchema:SchemaRequest|TypedSchema
Defined in: ai/lib/types/requests.ts:117
Output response schema of the generated candidate text. This
value can be a class generated with a Schema static method
like Schema.string() or Schema.object() or it can be a plain
JS object matching the SchemaRequest interface.
Note: This only applies when the specified responseMIMEType supports a schema; currently
this is limited to application/json and text/x.enum.
stopSequences?
optionalstopSequences:string[]
Defined in: ai/lib/types/requests.ts:95
temperature?
optionaltemperature:number
Defined in: ai/lib/types/requests.ts:97
thinkingConfig?
optionalthinkingConfig:ThinkingConfig
Defined in: ai/lib/types/requests.ts:131
Configuration for "thinking" behavior of compatible Gemini models.
topK?
optionaltopK:number
Defined in: ai/lib/types/requests.ts:99
topP?
optionaltopP:number
Defined in: ai/lib/types/requests.ts:98
GenerativeContentBlob
Defined in: ai/lib/types/content.ts:189
Interface for sending an image.
data
data:
string
Defined in: ai/lib/types/content.ts:194
Image as a base64 string.
mimeType
mimeType:
string
Defined in: ai/lib/types/content.ts:190
GoogleAICitationMetadata
Defined in: ai/lib/types/googleai.ts:68
Internal
citationSources
citationSources:
Citation[]
Defined in: ai/lib/types/googleai.ts:69
GoogleAICountTokensRequest
Defined in: ai/lib/types/googleai.ts:33
Internal
generateContentRequest
generateContentRequest:
object
Defined in: ai/lib/types/googleai.ts:34
contents
contents:
Content[]
generationConfig?
optionalgenerationConfig:GenerationConfig
tools?
optionaltools:Tool[]
GoogleAIGenerateContentCandidate
Defined in: ai/lib/types/googleai.ts:55
Internal
citationMetadata?
optionalcitationMetadata:GoogleAICitationMetadata
Defined in: ai/lib/types/googleai.ts:61
content
content:
Content
Defined in: ai/lib/types/googleai.ts:57
finishMessage?
optionalfinishMessage:string
Defined in: ai/lib/types/googleai.ts:59
finishReason?
optionalfinishReason:FinishReason
Defined in: ai/lib/types/googleai.ts:58
groundingMetadata?
optionalgroundingMetadata:GroundingMetadata
Defined in: ai/lib/types/googleai.ts:62
index
index:
number
Defined in: ai/lib/types/googleai.ts:56
safetyRatings?
optionalsafetyRatings:SafetyRating[]
Defined in: ai/lib/types/googleai.ts:60
GoogleAIGenerateContentResponse
Defined in: ai/lib/types/googleai.ts:46
Internal
candidates?
optionalcandidates:GoogleAIGenerateContentCandidate[]
Defined in: ai/lib/types/googleai.ts:47
promptFeedback?
optionalpromptFeedback:PromptFeedback
Defined in: ai/lib/types/googleai.ts:48
usageMetadata?
optionalusageMetadata:UsageMetadata
Defined in: ai/lib/types/googleai.ts:49
GoogleSearch
Defined in: ai/lib/types/requests.ts:317
Specifies the Google Search configuration.
GoogleSearchTool
Defined in: ai/lib/types/requests.ts:295
A tool that allows a Gemini model to connect to Google Search to access and incorporate up-to-date information from the web into its responses.
Important: If using Grounding with Google Search, you are required to comply with the "Grounding with Google Search" usage requirements for your chosen API provider: Gemini Developer API or Vertex AI Gemini API (see Service Terms section within the Service Specific Terms).
googleSearch
googleSearch:
GoogleSearch
Defined in: ai/lib/types/requests.ts:306
Specifies the Google Search configuration. Currently, this is an empty object, but it's reserved for future configuration options. Specifies the Google Search configuration.
When using this feature, you are required to comply with the "Grounding with Google Search" usage requirements for your chosen API provider: Gemini Developer API or Vertex AI Gemini API (see Service Terms section within the Service Specific Terms).
GroundingAttribution
Defined in: ai/lib/types/responses.ts:296
confidenceScore?
optionalconfidenceScore:number
Defined in: ai/lib/types/responses.ts:298
retrievedContext?
optionalretrievedContext:RetrievedContextAttribution
Defined in: ai/lib/types/responses.ts:300
segment
segment:
Segment
Defined in: ai/lib/types/responses.ts:297
web?
optionalweb:WebAttribution
Defined in: ai/lib/types/responses.ts:299
GroundingChunk
Defined in: ai/lib/types/responses.ts:237
Represents a chunk of retrieved data that supports a claim in the model's response. This is part of the grounding information provided when grounding is enabled.
web?
optionalweb:WebGroundingChunk
Defined in: ai/lib/types/responses.ts:241
Contains details if the grounding chunk is from a web source.
GroundingMetadata
Defined in: ai/lib/types/responses.ts:209
Metadata returned when grounding is enabled.
Currently, only Grounding with Google Search is supported (see GoogleSearchTool).
Important: If using Grounding with Google Search, you are required to comply with the "Grounding with Google Search" usage requirements for your chosen API provider: Gemini Developer API or Vertex AI Gemini API (see Service Terms section within the Service Specific Terms).
groundingChunks?
optionalgroundingChunks:GroundingChunk[]
Defined in: ai/lib/types/responses.ts:214
A list of GroundingChunk objects. Each chunk represents a piece of retrieved content (for example, from a web page). that the model used to ground its response.
groundingSupports?
optionalgroundingSupports:GroundingSupport[]
Defined in: ai/lib/types/responses.ts:219
A list of GroundingSupport objects. Each object details how specific segments of the
model's response are supported by the groundingChunks.
retrievalQueries?
optionalretrievalQueries:string[]
Defined in: ai/lib/types/responses.ts:228
Deprecated
Use GroundingSupport instead.
webSearchQueries?
optionalwebSearchQueries:string[]
Defined in: ai/lib/types/responses.ts:224
A list of web search queries that the model performed to gather the grounding information. These can be used to allow users to explore the search results themselves.
GroundingSupport
Defined in: ai/lib/types/responses.ts:277
Provides information about how a specific segment of the model's response is supported by the retrieved grounding chunks.
groundingChunkIndices?
optionalgroundingChunkIndices:number[]
Defined in: ai/lib/types/responses.ts:289
A list of indices that refer to specific GroundingChunk objects within the
GroundingMetadata.groundingChunks array. These referenced chunks
are the sources that support the claim made in the associated segment of the response.
For example, an array [1, 3, 4] means that groundingChunks[1], groundingChunks[3],
and groundingChunks[4] are the retrieved content supporting this part of the response.
segment?
optionalsegment:Segment
Defined in: ai/lib/types/responses.ts:281
Specifies the segment of the model's response content that this grounding support pertains to.
ImagenGCSImage
Defined in: ai/lib/types/imagen/responses.ts:42
Beta
An image generated by Imagen, stored in a Cloud Storage for Firebase bucket.
This feature is not available yet.
gcsURI
gcsURI:
string
Defined in: ai/lib/types/imagen/responses.ts:54
Beta
The URI of the file stored in a Cloud Storage for Firebase bucket.
mimeType
mimeType:
string
Defined in: ai/lib/types/imagen/responses.ts:48
Beta
The MIME type of the image; either "image/png" or "image/jpeg".
To request a different format, set the imageFormat property in your ImagenGenerationConfig.
ImagenGenerationConfig
Defined in: ai/lib/types/imagen/requests.ts:54
Beta
Configuration options for generating images with Imagen.
See the documentation for more details.
addWatermark?
optionaladdWatermark:boolean
Defined in: ai/lib/types/imagen/requests.ts:98
Beta
Whether to add an invisible watermark to generated images.
If set to true, an invisible SynthID watermark is embedded in generated images to indicate
that they are AI generated. If set to false, watermarking will be disabled.
For Imagen 3 models, the default value is true; see the addWatermark
documentation for more details.
When using the Gemini Developer API (GoogleAIBackend), this will default to true, and cannot be turned off.
aspectRatio?
optionalaspectRatio:ImagenAspectRatio
Defined in: ai/lib/types/imagen/requests.ts:79
Beta
The aspect ratio of the generated images. The default value is square 1:1. Supported aspect ratios depend on the Imagen model, see (ImagenAspectRatio:type) for more details.
imageFormat?
optionalimageFormat:ImagenImageFormat
Defined in: ai/lib/types/imagen/requests.ts:85
Beta
The image format of the generated images. The default is PNG.
See ImagenImageFormat for more details.
negativePrompt?
optionalnegativePrompt:string
Defined in: ai/lib/types/imagen/requests.ts:65
Beta
A description of what should be omitted from the generated images.
Support for negative prompts depends on the Imagen model.
See the documentation for more details.
This is no longer supported in the Gemini Developer API (GoogleAIBackend) in versions
greater than imagen-3.0-generate-002.
numberOfImages?
optionalnumberOfImages:number
Defined in: ai/lib/types/imagen/requests.ts:73
Beta
The number of images to generate. The default value is 1.
The number of sample images that may be generated in each request depends on the model (typically up to 4); see the sampleCount documentation for more details.
ImagenGenerationResponse
Defined in: ai/lib/types/imagen/responses.ts:62
Beta
The response from a request to generate images with Imagen.
T
T extends ImagenInlineImage | ImagenGCSImage
filteredReason?
optionalfilteredReason:string
Defined in: ai/lib/types/imagen/responses.ts:81
Beta
The reason that images were filtered out. This property will only be defined if one or more images were filtered.
Images may be filtered out due to the (ImagenSafetyFilterLevel:type), (ImagenPersonFilterLevel:type), or filtering included in the model. The filter levels may be adjusted in your ImagenSafetySettings.
See the Responsible AI and usage guidelines for Imagen for more details.
images
images:
T[]
Defined in: ai/lib/types/imagen/responses.ts:69
Beta
The images generated by Imagen.
The number of images generated may be fewer than the number requested if one or more were
filtered out; see filteredReason.
ImagenInlineImage
Defined in: ai/lib/types/imagen/responses.ts:23
Beta
An image generated by Imagen, represented as inline data.
bytesBase64Encoded
bytesBase64Encoded:
string
Defined in: ai/lib/types/imagen/responses.ts:33
Beta
The base64-encoded image data.
mimeType
mimeType:
string
Defined in: ai/lib/types/imagen/responses.ts:29
Beta
The MIME type of the image; either "image/png" or "image/jpeg".
To request a different format, set the imageFormat property in your ImagenGenerationConfig.
ImagenModelParams
Defined in: ai/lib/types/imagen/requests.ts:25
Beta
Parameters for configuring an ImagenModel.
generationConfig?
optionalgenerationConfig:ImagenGenerationConfig
Defined in: ai/lib/types/imagen/requests.ts:39
Beta
Configuration options for generating images with Imagen.
model
model:
string
Defined in: ai/lib/types/imagen/requests.ts:35
Beta
The Imagen model to use for generating images.
For example: imagen-3.0-generate-002.
Only Imagen 3 models (named imagen-3.0-*) are supported.
See model versions for a full list of supported Imagen 3 models.
safetySettings?
optionalsafetySettings:ImagenSafetySettings
Defined in: ai/lib/types/imagen/requests.ts:43
Beta
Safety settings for filtering potentially inappropriate content.
ImagenSafetySettings
Defined in: ai/lib/types/imagen/requests.ts:200
Beta
Settings for controlling the aggressiveness of filtering out sensitive content.
See the documentation for more details.
personFilterLevel?
optionalpersonFilterLevel:ImagenPersonFilterLevel
Defined in: ai/lib/types/imagen/requests.ts:209
Beta
A filter level controlling whether generation of images containing people or faces is allowed.
safetyFilterLevel?
optionalsafetyFilterLevel:ImagenSafetyFilterLevel
Defined in: ai/lib/types/imagen/requests.ts:205
Beta
A filter level controlling how aggressive to filter out sensitive content from generated images.
InlineDataPart
Defined in: ai/lib/types/content.ts:61
Content part interface if the part represents an image.
functionCall?
optionalfunctionCall:undefined
Defined in: ai/lib/types/content.ts:64
functionResponse?
optionalfunctionResponse:undefined
Defined in: ai/lib/types/content.ts:65
inlineData
inlineData:
GenerativeContentBlob
Defined in: ai/lib/types/content.ts:63
text?
optionaltext:undefined
Defined in: ai/lib/types/content.ts:62
thought?
optionalthought:boolean
Defined in: ai/lib/types/content.ts:70
thoughtSignature?
optionalthoughtSignature:undefined
Defined in: ai/lib/types/content.ts:74
Internal
videoMetadata?
optionalvideoMetadata:VideoMetadata
Defined in: ai/lib/types/content.ts:69
Applicable if inlineData is a video.
LiveGenerationConfig
Defined in: ai/lib/types/requests.ts:139
Beta
Configuration parameters used by LiveGenerativeModel to control live content generation.
frequencyPenalty?
optionalfrequencyPenalty:number
Defined in: ai/lib/types/requests.ts:178
Beta
Frequency penalties.
inputAudioTranscription?
optionalinputAudioTranscription:AudioTranscriptionConfig
Defined in: ai/lib/types/requests.ts:191
Beta
Enables transcription of audio input.
When enabled, the model will respond with transcriptions of your audio input in the inputTranscriptions property
in LiveServerContent messages. Note that the transcriptions are broken up across
messages, so you may only receive small amounts of text per message. For example, if you ask the model
"How are you today?", the model may transcribe that input across three messages, broken up as "How a", "re yo", "u today?".
maxOutputTokens?
optionalmaxOutputTokens:number
Defined in: ai/lib/types/requests.ts:148
Beta
Specifies the maximum number of tokens that can be generated in the response. The number of tokens per word varies depending on the language outputted. Is unbounded by default.
outputAudioTranscription?
optionaloutputAudioTranscription:AudioTranscriptionConfig
Defined in: ai/lib/types/requests.ts:200
Beta
Enables transcription of audio input.
When enabled, the model will respond with transcriptions of its audio output in the outputTranscription property
in LiveServerContent messages. Note that the transcriptions are broken up across
messages, so you may only receive small amounts of text per message. For example, if the model says
"How are you today?", the model may transcribe that output across three messages, broken up as "How a", "re yo", "u today?".
presencePenalty?
optionalpresencePenalty:number
Defined in: ai/lib/types/requests.ts:174
Beta
Positive penalties.
responseModalities?
optionalresponseModalities:ResponseModality[]
Defined in: ai/lib/types/requests.ts:182
Beta
The modalities of the response.
speechConfig?
optionalspeechConfig:SpeechConfig
Defined in: ai/lib/types/requests.ts:143
Beta
Configuration for speech synthesis.
temperature?
optionaltemperature:number
Defined in: ai/lib/types/requests.ts:154
Beta
Controls the degree of randomness in token selection. A temperature value of 0 means that the highest
probability tokens are always selected. In this case, responses for a given prompt are mostly
deterministic, but a small amount of variation is still possible.
topK?
optionaltopK:number
Defined in: ai/lib/types/requests.ts:170
Beta
Changes how the model selects token for output. A topK value of 1 means the select token is
the most probable among all tokens in the model's vocabulary, while a topK value 3 means that
the next token is selected from among the 3 most probably using probabilities sampled. Tokens
are then further filtered with the highest selected temperature sampling. Defaults to 40
if unspecified.
topP?
optionaltopP:number
Defined in: ai/lib/types/requests.ts:162
Beta
Changes how the model selects tokens for output. Tokens are
selected from the most to least probable until the sum of their probabilities equals the topP
value. For example, if tokens A, B, and C have probabilities of 0.3, 0.2, and 0.1 respectively
and the topP value is 0.5, then the model will select either A or B as the next token by using
the temperature and exclude C as a candidate. Defaults to 0.95 if unset.
LiveModelParams
Defined in: ai/lib/types/requests.ts:53
Beta
Params passed to getLiveGenerativeModel.
generationConfig?
optionalgenerationConfig:LiveGenerationConfig
Defined in: ai/lib/types/requests.ts:55
Beta
model
model:
string
Defined in: ai/lib/types/requests.ts:54
Beta
systemInstruction?
Defined in: ai/lib/types/requests.ts:58
Beta
toolConfig?
optionaltoolConfig:ToolConfig
Defined in: ai/lib/types/requests.ts:57
Beta
tools?
optionaltools:Tool[]
Defined in: ai/lib/types/requests.ts:56
Beta
LiveServerContent
Defined in: ai/lib/types/responses.ts:421
Beta
An incremental content update from the model.
inputTranscription?
optionalinputTranscription:Transcription
Defined in: ai/lib/types/responses.ts:440
Beta
Transcription of the audio that was input to the model.
interrupted?
optionalinterrupted:boolean
Defined in: ai/lib/types/responses.ts:436
Beta
Indicates whether the model was interrupted by the client. An interruption occurs when
the client sends a message before the model finishes it's turn. This is undefined if the
model was not interrupted.
modelTurn?
optionalmodelTurn:Content
Defined in: ai/lib/types/responses.ts:426
Beta
The content that the model has generated as part of the current conversation with the user.
outputTranscription?
optionaloutputTranscription:Transcription
Defined in: ai/lib/types/responses.ts:444
Beta
Transcription of the audio output from the model.
turnComplete?
optionalturnComplete:boolean
Defined in: ai/lib/types/responses.ts:430
Beta
Indicates whether the turn is complete. This is undefined if the turn is not complete.
type
type:
"serverContent"
Defined in: ai/lib/types/responses.ts:422
Beta
LiveServerToolCall
Defined in: ai/lib/types/responses.ts:467
Beta
A request from the model for the client to execute one or more functions.
functionCalls
functionCalls:
FunctionCall[]
Defined in: ai/lib/types/responses.ts:472
Beta
An array of function calls to run.
type
type:
"toolCall"
Defined in: ai/lib/types/responses.ts:468
Beta
LiveServerToolCallCancellation
Defined in: ai/lib/types/responses.ts:480
Beta
Notification to cancel a previous function call triggered by LiveServerToolCall.
functionIds
functionIds:
string[]
Defined in: ai/lib/types/responses.ts:485
Beta
IDs of function calls that were cancelled. These refer to the id property of a FunctionCall.
type
type:
"toolCallCancellation"
Defined in: ai/lib/types/responses.ts:481
Beta
ModalityTokenCount
Defined in: ai/lib/types/responses.ts:129
Represents token counting info for a single modality.
modality
modality:
Modality
Defined in: ai/lib/types/responses.ts:131
The modality associated with this token count.
tokenCount
tokenCount:
number
Defined in: ai/lib/types/responses.ts:133
The number of tokens counted.
ModelParams
Defined in: ai/lib/types/requests.ts:42
Params passed to getGenerativeModel.
generationConfig?
optionalgenerationConfig:GenerationConfig
Defined in: ai/lib/types/requests.ts:35
model
model:
string
Defined in: ai/lib/types/requests.ts:43
safetySettings?
optionalsafetySettings:SafetySetting[]
Defined in: ai/lib/types/requests.ts:34
systemInstruction?
Defined in: ai/lib/types/requests.ts:46
toolConfig?
optionaltoolConfig:ToolConfig
Defined in: ai/lib/types/requests.ts:45
tools?
optionaltools:Tool[]
Defined in: ai/lib/types/requests.ts:44
ObjectSchemaInterface
Defined in: ai/lib/types/schema.ts:121
Interface for ObjectSchema class.
description?
optionaldescription:string
Defined in: ai/lib/types/schema.ts:51
Optional. The description of the property.
enum?
optionalenum:string[]
Defined in: ai/lib/types/schema.ts:71
Optional. The enum of the property.
example?
optionalexample:unknown
Defined in: ai/lib/types/schema.ts:73
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/types/schema.ts:49
Optional. The format of the property.
When using the Gemini Developer API (GoogleAIBackend), this must be either 'enum' or
'date-time', otherwise requests will fail.
items?
optionalitems:SchemaInterface
Defined in: ai/lib/types/schema.ts:59
Optional. The items of the property.
maximum?
optionalmaximum:number
Defined in: ai/lib/types/schema.ts:79
The maximum value of a numeric type.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/types/schema.ts:63
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minimum?
optionalminimum:number
Defined in: ai/lib/types/schema.ts:77
The minimum value of a numeric type.
minItems?
optionalminItems:number
Defined in: ai/lib/types/schema.ts:61
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable?
optionalnullable:boolean
Defined in: ai/lib/types/schema.ts:75
Optional. Whether the property is nullable.
optionalProperties?
optionaloptionalProperties:string[]
Defined in: ai/lib/types/schema.ts:123
properties?
optionalproperties:object
Defined in: ai/lib/types/schema.ts:65
Optional. Map of Schema objects.
Index Signature
[k: string]: SchemaInterface
propertyOrdering?
optionalpropertyOrdering:string[]
Defined in: ai/lib/types/schema.ts:69
A hint suggesting the order in which the keys should appear in the generated JSON string.
title?
optionaltitle:string
Defined in: ai/lib/types/schema.ts:57
The title of the property. This helps document the schema's purpose but does not typically constrain the generated value. It can subtly guide the model by clarifying the intent of a field.
type
type:
OBJECT
Defined in: ai/lib/types/schema.ts:122
The type of the property. SchemaType.
PrebuiltVoiceConfig
Defined in: ai/lib/types/requests.ts:395
Beta
Configuration for a pre-built voice.
voiceName?
optionalvoiceName:string
Defined in: ai/lib/types/requests.ts:401
Beta
The voice name to use for speech synthesis.
For a full list of names and demos of what each voice sounds like, see Chirp 3: HD Voices.
PromptFeedback
Defined in: ai/lib/types/responses.ts:141
If the prompt was blocked, this will be populated with blockReason and
the relevant safetyRatings.
blockReason?
optionalblockReason:BlockReason
Defined in: ai/lib/types/responses.ts:142
blockReasonMessage?
optionalblockReasonMessage:string
Defined in: ai/lib/types/responses.ts:149
A human-readable description of the blockReason.
This property is only supported in the Vertex AI Gemini API (VertexAIBackend).
safetyRatings
safetyRatings:
SafetyRating[]
Defined in: ai/lib/types/responses.ts:143
RequestOptions
Defined in: ai/lib/types/requests.ts:238
Params passed to getGenerativeModel.
baseUrl?
optionalbaseUrl:string
Defined in: ai/lib/types/requests.ts:246
Base url for endpoint. Defaults to https://firebasevertexai.googleapis.com
timeout?
optionaltimeout:number
Defined in: ai/lib/types/requests.ts:242
Request timeout in milliseconds. Defaults to 180 seconds (180000ms).
RetrievedContextAttribution
Defined in: ai/lib/types/responses.ts:345
title
title:
string
Defined in: ai/lib/types/responses.ts:347
uri
uri:
string
Defined in: ai/lib/types/responses.ts:346
SafetyRating
Defined in: ai/lib/types/responses.ts:364
A safety rating associated with a GenerateContentCandidate
blocked
blocked:
boolean
Defined in: ai/lib/types/responses.ts:388
category
category:
HarmCategory
Defined in: ai/lib/types/responses.ts:365
probability
probability:
HarmProbability
Defined in: ai/lib/types/responses.ts:366
probabilityScore
probabilityScore:
number
Defined in: ai/lib/types/responses.ts:380
The probability score of the harm category.
This property is only supported when using the Vertex AI Gemini API (VertexAIBackend). When using the Gemini Developer API (GoogleAIBackend), this property is not supported and will default to 0.
severity
severity:
HarmSeverity
Defined in: ai/lib/types/responses.ts:373
The harm severity level.
This property is only supported when using the Vertex AI Gemini API (VertexAIBackend).
When using the Gemini Developer API (GoogleAIBackend), this property is not supported and will default to HarmSeverity.UNSUPPORTED.
severityScore
severityScore:
number
Defined in: ai/lib/types/responses.ts:387
The severity score of the harm category.
This property is only supported when using the Vertex AI Gemini API (VertexAIBackend). When using the Gemini Developer API (GoogleAIBackend), this property is not supported and will default to 0.
SafetySetting
Defined in: ai/lib/types/requests.ts:76
Safety setting that can be sent as part of request parameters.
category
category:
HarmCategory
Defined in: ai/lib/types/requests.ts:77
method?
optionalmethod:HarmBlockMethod
Defined in: ai/lib/types/requests.ts:86
The harm block method.
This property is only supported in the Vertex AI Gemini API (VertexAIBackend). When using the Gemini Developer API (GoogleAIBackend), an AIError will be thrown if this property is defined.
threshold
threshold:
HarmBlockThreshold
Defined in: ai/lib/types/requests.ts:78
SchemaInterface
Defined in: ai/lib/types/schema.ts:109
Interface for Schema class.
description?
optionaldescription:string
Defined in: ai/lib/types/schema.ts:51
Optional. The description of the property.
enum?
optionalenum:string[]
Defined in: ai/lib/types/schema.ts:71
Optional. The enum of the property.
example?
optionalexample:unknown
Defined in: ai/lib/types/schema.ts:73
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/types/schema.ts:49
Optional. The format of the property.
When using the Gemini Developer API (GoogleAIBackend), this must be either 'enum' or
'date-time', otherwise requests will fail.
items?
optionalitems:SchemaInterface
Defined in: ai/lib/types/schema.ts:59
Optional. The items of the property.
maximum?
optionalmaximum:number
Defined in: ai/lib/types/schema.ts:79
The maximum value of a numeric type.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/types/schema.ts:63
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minimum?
optionalminimum:number
Defined in: ai/lib/types/schema.ts:77
The minimum value of a numeric type.
minItems?
optionalminItems:number
Defined in: ai/lib/types/schema.ts:61
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable?
optionalnullable:boolean
Defined in: ai/lib/types/schema.ts:75
Optional. Whether the property is nullable.
properties?
optionalproperties:object
Defined in: ai/lib/types/schema.ts:65
Optional. Map of Schema objects.
Index Signature
[k: string]: SchemaInterface
propertyOrdering?
optionalpropertyOrdering:string[]
Defined in: ai/lib/types/schema.ts:69
A hint suggesting the order in which the keys should appear in the generated JSON string.
title?
optionaltitle:string
Defined in: ai/lib/types/schema.ts:57
The title of the property. This helps document the schema's purpose but does not typically constrain the generated value. It can subtly guide the model by clarifying the intent of a field.
type
type:
SchemaType
Defined in: ai/lib/types/schema.ts:114
The type of the property. SchemaType.
SchemaParams
Defined in: ai/lib/types/schema.ts:89
Params passed to Schema static methods to create specific Schema classes.
description?
optionaldescription:string
Defined in: ai/lib/types/schema.ts:51
Optional. The description of the property.
enum?
optionalenum:string[]
Defined in: ai/lib/types/schema.ts:71
Optional. The enum of the property.
example?
optionalexample:unknown
Defined in: ai/lib/types/schema.ts:73
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/types/schema.ts:49
Optional. The format of the property.
When using the Gemini Developer API (GoogleAIBackend), this must be either 'enum' or
'date-time', otherwise requests will fail.
items?
optionalitems:SchemaInterface
Defined in: ai/lib/types/schema.ts:59
Optional. The items of the property.
maximum?
optionalmaximum:number
Defined in: ai/lib/types/schema.ts:79
The maximum value of a numeric type.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/types/schema.ts:63
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minimum?
optionalminimum:number
Defined in: ai/lib/types/schema.ts:77
The minimum value of a numeric type.
minItems?
optionalminItems:number
Defined in: ai/lib/types/schema.ts:61
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable?
optionalnullable:boolean
Defined in: ai/lib/types/schema.ts:75
Optional. Whether the property is nullable.
properties?
optionalproperties:object
Defined in: ai/lib/types/schema.ts:65
Optional. Map of Schema objects.
Index Signature
[k: string]: SchemaInterface
propertyOrdering?
optionalpropertyOrdering:string[]
Defined in: ai/lib/types/schema.ts:69
A hint suggesting the order in which the keys should appear in the generated JSON string.
title?
optionaltitle:string
Defined in: ai/lib/types/schema.ts:57
The title of the property. This helps document the schema's purpose but does not typically constrain the generated value. It can subtly guide the model by clarifying the intent of a field.
SchemaRequest
Defined in: ai/lib/types/schema.ts:95
Final format for Schema params passed to backend requests.
description?
optionaldescription:string
Defined in: ai/lib/types/schema.ts:51
Optional. The description of the property.
enum?
optionalenum:string[]
Defined in: ai/lib/types/schema.ts:71
Optional. The enum of the property.
example?
optionalexample:unknown
Defined in: ai/lib/types/schema.ts:73
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/types/schema.ts:49
Optional. The format of the property.
When using the Gemini Developer API (GoogleAIBackend), this must be either 'enum' or
'date-time', otherwise requests will fail.
items?
optionalitems:SchemaRequest
Defined in: ai/lib/types/schema.ts:59
Optional. The items of the property.
maximum?
optionalmaximum:number
Defined in: ai/lib/types/schema.ts:79
The maximum value of a numeric type.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/types/schema.ts:63
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minimum?
optionalminimum:number
Defined in: ai/lib/types/schema.ts:77
The minimum value of a numeric type.
minItems?
optionalminItems:number
Defined in: ai/lib/types/schema.ts:61
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable?
optionalnullable:boolean
Defined in: ai/lib/types/schema.ts:75
Optional. Whether the property is nullable.
properties?
optionalproperties:object
Defined in: ai/lib/types/schema.ts:65
Optional. Map of Schema objects.
Index Signature
[k: string]: SchemaRequest
propertyOrdering?
optionalpropertyOrdering:string[]
Defined in: ai/lib/types/schema.ts:69
A hint suggesting the order in which the keys should appear in the generated JSON string.
required?
optionalrequired:string[]
Defined in: ai/lib/types/schema.ts:102
Optional. Array of required property.
title?
optionaltitle:string
Defined in: ai/lib/types/schema.ts:57
The title of the property. This helps document the schema's purpose but does not typically constrain the generated value. It can subtly guide the model by clarifying the intent of a field.
type
type:
SchemaType
Defined in: ai/lib/types/schema.ts:100
The type of the property. SchemaType.
SchemaShared
Defined in: ai/lib/types/schema.ts:44
Basic Schema properties shared across several Schema-related types.
description?
optionaldescription:string
Defined in: ai/lib/types/schema.ts:51
Optional. The description of the property.
enum?
optionalenum:string[]
Defined in: ai/lib/types/schema.ts:71
Optional. The enum of the property.
example?
optionalexample:unknown
Defined in: ai/lib/types/schema.ts:73
Optional. The example of the property.
format?
optionalformat:string
Defined in: ai/lib/types/schema.ts:49
Optional. The format of the property.
When using the Gemini Developer API (GoogleAIBackend), this must be either 'enum' or
'date-time', otherwise requests will fail.
items?
optionalitems:T
Defined in: ai/lib/types/schema.ts:59
Optional. The items of the property.
maximum?
optionalmaximum:number
Defined in: ai/lib/types/schema.ts:79
The maximum value of a numeric type.
maxItems?
optionalmaxItems:number
Defined in: ai/lib/types/schema.ts:63
The maximum number of items (elements) in a schema of type SchemaType.ARRAY.
minimum?
optionalminimum:number
Defined in: ai/lib/types/schema.ts:77
The minimum value of a numeric type.
minItems?
optionalminItems:number
Defined in: ai/lib/types/schema.ts:61
The minimum number of items (elements) in a schema of type SchemaType.ARRAY.
nullable?
optionalnullable:boolean
Defined in: ai/lib/types/schema.ts:75
Optional. Whether the property is nullable.
properties?
optionalproperties:object
Defined in: ai/lib/types/schema.ts:65
Optional. Map of Schema objects.
propertyOrdering?
optionalpropertyOrdering:string[]
Defined in: ai/lib/types/schema.ts:69
A hint suggesting the order in which the keys should appear in the generated JSON string.
title?
optionaltitle:string
Defined in: ai/lib/types/schema.ts:57
The title of the property. This helps document the schema's purpose but does not typically constrain the generated value. It can subtly guide the model by clarifying the intent of a field.
Segment
Defined in: ai/lib/types/responses.ts:309
Represents a specific segment within a Content object, often used to pinpoint the exact location of text or data that grounding information refers to.
endIndex
endIndex:
number
Defined in: ai/lib/types/responses.ts:327
The zero-based end index of the segment within the specified Part,
measured in UTF-8 bytes. This offset is exclusive, meaning the character
at this index is not included in the segment.
partIndex
partIndex:
number
Defined in: ai/lib/types/responses.ts:315
The zero-based index of the Part object within the parts array
of its parent Content object. This identifies which part of the
content the segment belongs to.
startIndex
startIndex:
number
Defined in: ai/lib/types/responses.ts:321
The zero-based start index of the segment within the specified Part,
measured in UTF-8 bytes. This offset is inclusive, starting from 0 at the
beginning of the part's content (e.g., Part.text).
text
text:
string
Defined in: ai/lib/types/responses.ts:331
The text corresponding to the segment from the response.
SpeechConfig
Defined in: ai/lib/types/requests.ts:421
Beta
Configures speech synthesis.
voiceConfig?
optionalvoiceConfig:VoiceConfig
Defined in: ai/lib/types/requests.ts:425
Beta
Configures the voice to be used in speech synthesis.
StartChatParams
Defined in: ai/lib/types/requests.ts:207
Params for GenerativeModel.startChat.
generationConfig?
optionalgenerationConfig:GenerationConfig
Defined in: ai/lib/types/requests.ts:35
history?
optionalhistory:Content[]
Defined in: ai/lib/types/requests.ts:208
safetySettings?
optionalsafetySettings:SafetySetting[]
Defined in: ai/lib/types/requests.ts:34
systemInstruction?
Defined in: ai/lib/types/requests.ts:211
toolConfig?
optionaltoolConfig:ToolConfig
Defined in: ai/lib/types/requests.ts:210
tools?
optionaltools:Tool[]
Defined in: ai/lib/types/requests.ts:209
TextPart
Defined in: ai/lib/types/content.ts:45
Content part interface if the part represents a text string.
functionCall?
optionalfunctionCall:undefined
Defined in: ai/lib/types/content.ts:48
functionResponse?
optionalfunctionResponse:undefined
Defined in: ai/lib/types/content.ts:49
inlineData?
optionalinlineData:undefined
Defined in: ai/lib/types/content.ts:47
text
text:
string
Defined in: ai/lib/types/content.ts:46
thought?
optionalthought:boolean
Defined in: ai/lib/types/content.ts:50
thoughtSignature?
optionalthoughtSignature:string
Defined in: ai/lib/types/content.ts:54
Internal
ThinkingConfig
Defined in: ai/lib/types/requests.ts:363
Configuration for "thinking" behavior of compatible Gemini models.
Certain models utilize a thinking process before generating a response. This allows them to reason through complex problems and plan a more coherent and accurate answer.
includeThoughts?
optionalincludeThoughts:boolean
Defined in: ai/lib/types/requests.ts:387
Whether to include "thought summaries" in the model's response.
Remarks
Thought summaries provide a brief overview of the model's internal thinking process, offering insight into how it arrived at the final answer. This can be useful for debugging, understanding the model's reasoning, and verifying its accuracy.
thinkingBudget?
optionalthinkingBudget:number
Defined in: ai/lib/types/requests.ts:377
The thinking budget, in tokens.
This parameter sets an upper limit on the number of tokens the model can use for its internal "thinking" process. A higher budget may result in higher quality responses for complex tasks but can also increase latency and cost.
If you don't specify a budget, the model will determine the appropriate amount of thinking based on the complexity of the prompt.
An error will be thrown if you set a thinking budget for a model that does not support this feature or if the specified budget is not within the model's supported range.
ToolConfig
Defined in: ai/lib/types/requests.ts:343
Tool config. This config is shared for all tools provided in the request.
functionCallingConfig?
optionalfunctionCallingConfig:FunctionCallingConfig
Defined in: ai/lib/types/requests.ts:344
Transcription
Defined in: ai/lib/types/responses.ts:455
Beta
Transcription of audio. This can be returned from a LiveGenerativeModel if transcription
is enabled with the inputAudioTranscription or outputAudioTranscription properties on
the LiveGenerationConfig.
text?
optionaltext:string
Defined in: ai/lib/types/responses.ts:459
Beta
The text transcription of the audio.
UsageMetadata
Defined in: ai/lib/types/responses.ts:112
Usage metadata about a GenerateContentResponse.
candidatesTokenCount
candidatesTokenCount:
number
Defined in: ai/lib/types/responses.ts:114
candidatesTokensDetails?
optionalcandidatesTokensDetails:ModalityTokenCount[]
Defined in: ai/lib/types/responses.ts:121
promptTokenCount
promptTokenCount:
number
Defined in: ai/lib/types/responses.ts:113
promptTokensDetails?
optionalpromptTokensDetails:ModalityTokenCount[]
Defined in: ai/lib/types/responses.ts:120
thoughtsTokenCount?
optionalthoughtsTokenCount:number
Defined in: ai/lib/types/responses.ts:118
The number of tokens used by the model's internal "thinking" process.
totalTokenCount
totalTokenCount:
number
Defined in: ai/lib/types/responses.ts:119
VideoMetadata
Defined in: ai/lib/types/content.ts:81
Describes the input video content.
endOffset
endOffset:
string
Defined in: ai/lib/types/content.ts:91
The end offset of the video in protobuf Duration format.
startOffset
startOffset:
string
Defined in: ai/lib/types/content.ts:86
The start offset of the video in protobuf Duration format.
VoiceConfig
Defined in: ai/lib/types/requests.ts:409
Beta
Configuration for the voice to used in speech synthesis.
prebuiltVoiceConfig?
optionalprebuiltVoiceConfig:PrebuiltVoiceConfig
Defined in: ai/lib/types/requests.ts:413
Beta
Configures the voice using a pre-built voice configuration.
WebAttribution
Defined in: ai/lib/types/responses.ts:337
title
title:
string
Defined in: ai/lib/types/responses.ts:339
uri
uri:
string
Defined in: ai/lib/types/responses.ts:338
WebGroundingChunk
Defined in: ai/lib/types/responses.ts:252
A grounding chunk from the web.
Important: If using Grounding with Google Search, you are required to comply with the Service Specific Terms for "Grounding with Google Search".
domain?
optionaldomain:string
Defined in: ai/lib/types/responses.ts:268
The domain of the original URI from which the content was retrieved.
This property is only supported in the Vertex AI Gemini API (VertexAIBackend).
When using the Gemini Developer API (GoogleAIBackend), this property will be
undefined.
title?
optionaltitle:string
Defined in: ai/lib/types/responses.ts:260
The title of the retrieved web page.
uri?
optionaluri:string
Defined in: ai/lib/types/responses.ts:256
The URI of the retrieved web page.
BackendType
BackendType = typeof
BackendType[keyof typeofBackendType]
Defined in: ai/lib/public-types.ts:59
Type alias representing valid backend types.
It can be either 'VERTEX_AI' or 'GOOGLE_AI'.
ImagenAspectRatio
ImagenAspectRatio = typeof
ImagenAspectRatio[keyof typeofImagenAspectRatio]
Defined in: ai/lib/types/imagen/requests.ts:223
Beta
Aspect ratios for Imagen images.
To specify an aspect ratio for generated images, set the aspectRatio property in your
ImagenGenerationConfig.
See the documentation for more details and examples of the supported aspect ratios.
ImagenPersonFilterLevel
ImagenPersonFilterLevel = typeof
ImagenPersonFilterLevel[keyof typeofImagenPersonFilterLevel]
Defined in: ai/lib/types/imagen/requests.ts:158
Beta
A filter level controlling whether generation of images containing people or faces is allowed.
See the personGeneration documentation for more details.
ImagenSafetyFilterLevel
ImagenSafetyFilterLevel = typeof
ImagenSafetyFilterLevel[keyof typeofImagenSafetyFilterLevel]
Defined in: ai/lib/types/imagen/requests.ts:113
Beta
A filter level controlling how aggressively to filter sensitive content.
Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI
are assessed against a list of safety filters, which include 'harmful categories' (for example,
violence, sexual, derogatory, and toxic). This filter level controls how aggressively to
filter out potentially harmful content from responses. See the documentation
and the Responsible AI and usage guidelines
for more details.
InferenceMode
InferenceMode = typeof
InferenceMode[keyof typeofInferenceMode]
Defined in: ai/lib/types/enums.ts:295
(EXPERIMENTAL) Determines whether inference happens on-device or in-cloud.
LiveResponseType
LiveResponseType = typeof
LiveResponseType[keyof typeofLiveResponseType]
Defined in: ai/lib/types/responses.ts:493
Beta
The types of responses that can be returned by LiveSession.receive. This is a property on all messages that can be used for type narrowing. This property is not returned by the server, it is assigned to a server message object once it's parsed.
Part
Part =
TextPart|InlineDataPart|FunctionCallPart|FunctionResponsePart|FileDataPart
Defined in: ai/lib/types/content.ts:34
Content part - includes text, image/video, or function call/response part types.
ResponseModality
ResponseModality = typeof
ResponseModality[keyof typeofResponseModality]
Defined in: ai/lib/types/enums.ts:265
Beta
Generation modalities to be returned in generation responses.
Role
Role = typeof
POSSIBLE_ROLES[number]
Defined in: ai/lib/types/enums.ts:22
Role is the producer of the content.
Tool
Tool =
FunctionDeclarationsTool|GoogleSearchTool
Defined in: ai/lib/types/requests.ts:253
Defines a tool that model can call to access external knowledge.
TypedSchema
TypedSchema =
IntegerSchema|NumberSchema|StringSchema|BooleanSchema|ObjectSchema|ArraySchema
Defined in: ai/lib/requests/schema-builder.ts:136
A type that includes all specific Schema types.
BackendType
constBackendType:object
Defined in: ai/lib/public-types.ts:59
An enum-like object containing constants that represent the supported backends for the Firebase AI SDK. This determines which backend service (Vertex AI Gemini API or Gemini Developer API) the SDK will communicate with.
These values are assigned to the backendType property within the specific backend
configuration objects (GoogleAIBackend or VertexAIBackend) to identify
which service to target.
GOOGLE_AI
readonlyGOOGLE_AI:"GOOGLE_AI"='GOOGLE_AI'
Identifies the backend service for the Gemini Developer API (Google AI). Use this constant when creating a GoogleAIBackend configuration.
VERTEX_AI
readonlyVERTEX_AI:"VERTEX_AI"='VERTEX_AI'
Identifies the backend service for the Vertex AI Gemini API provided through Google Cloud. Use this constant when creating a VertexAIBackend configuration.
ImagenAspectRatio
constImagenAspectRatio:object
Defined in: ai/lib/types/imagen/requests.ts:223
Beta
Aspect ratios for Imagen images.
To specify an aspect ratio for generated images, set the aspectRatio property in your
ImagenGenerationConfig.
See the documentation for more details and examples of the supported aspect ratios.
ImagenPersonFilterLevel
constImagenPersonFilterLevel:object
Defined in: ai/lib/types/imagen/requests.ts:158
Beta
A filter level controlling whether generation of images containing people or faces is allowed.
See the personGeneration documentation for more details.
ALLOW_ADULT
readonlyALLOW_ADULT:"allow_adult"='allow_adult'
Allow generation of images containing adults only; images of children are filtered out.
Generation of images containing people or faces may require your use case to be reviewed and approved by Cloud support; see the Responsible AI and usage guidelines for more details.
ALLOW_ALL
readonlyALLOW_ALL:"allow_all"='allow_all'
Allow generation of images containing adults only; images of children are filtered out.
Generation of images containing people or faces may require your use case to be reviewed and approved by Cloud support; see the Responsible AI and usage guidelines for more details.
BLOCK_ALL
readonlyBLOCK_ALL:"dont_allow"='dont_allow'
Disallow generation of images containing people or faces; images of people are filtered out.
ImagenSafetyFilterLevel
constImagenSafetyFilterLevel:object
Defined in: ai/lib/types/imagen/requests.ts:113
Beta
A filter level controlling how aggressively to filter sensitive content.
Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI
are assessed against a list of safety filters, which include 'harmful categories' (for example,
violence, sexual, derogatory, and toxic). This filter level controls how aggressively to
filter out potentially harmful content from responses. See the documentation
and the Responsible AI and usage guidelines
for more details.
BLOCK_LOW_AND_ABOVE
readonlyBLOCK_LOW_AND_ABOVE:"block_low_and_above"='block_low_and_above'
The most aggressive filtering level; most strict blocking.
BLOCK_MEDIUM_AND_ABOVE
readonlyBLOCK_MEDIUM_AND_ABOVE:"block_medium_and_above"='block_medium_and_above'
Blocks some sensitive prompts and responses.
BLOCK_NONE
readonlyBLOCK_NONE:"block_none"='block_none'
The least aggressive filtering level; blocks very few sensitive prompts and responses.
Access to this feature is restricted and may require your case to be reviewed and approved by Cloud support.
BLOCK_ONLY_HIGH
readonlyBLOCK_ONLY_HIGH:"block_only_high"='block_only_high'
Blocks few sensitive prompts and responses.
InferenceMode
constInferenceMode:object
Defined in: ai/lib/types/enums.ts:295
(EXPERIMENTAL) Determines whether inference happens on-device or in-cloud.
LiveResponseType
constLiveResponseType:object
Defined in: ai/lib/types/responses.ts:493
Beta
The types of responses that can be returned by LiveSession.receive.
POSSIBLE_ROLES
constPOSSIBLE_ROLES: readonly ["user","model","function","system"]
Defined in: ai/lib/types/enums.ts:28
Possible roles.
ResponseModality
constResponseModality:object
Defined in: ai/lib/types/enums.ts:265
Beta
Generation modalities to be returned in generation responses.
getAI()
getAI(
app?,options?):AI
Defined in: ai/lib/index.ts:78
Returns the default AI instance that is associated with the provided @firebase/app#FirebaseApp. If no instance exists, initializes a new instance with the default settings.
Examples
const ai = getAI(app);
// Get an AI instance configured to use the Gemini Developer API (via Google AI).
const ai = getAI(app, { backend: new GoogleAIBackend() });
// Get an AI instance configured to use the Vertex AI Gemini API.
const ai = getAI(app, { backend: new VertexAIBackend() });
getGenerativeModel()
getGenerativeModel(
ai,modelParams,requestOptions?):GenerativeModel
Defined in: ai/lib/index.ts:103
Returns a GenerativeModel class with methods for inference and other functionality.
getImagenModel()
getImagenModel(
ai,modelParams,requestOptions?):ImagenModel
Defined in: ai/lib/index.ts:131
Beta
Returns an ImagenModel class with methods for using Imagen.
Only Imagen 3 models (named imagen-3.0-*) are supported.
getLiveGenerativeModel()
getLiveGenerativeModel(
ai,modelParams):LiveGenerativeModel
Defined in: ai/lib/index.ts:157
Beta
Returns a LiveGenerativeModel class for real-time, bidirectional communication.
The Live API is only supported in modern browser windows and Node >= 22.
modelParams
Parameters to use when setting up a LiveSession.
getTemplateGenerativeModel()
getTemplateGenerativeModel(
ai,requestOptions?):TemplateGenerativeModel
Defined in: ai/lib/index.ts:176
Beta
Returns a TemplateGenerativeModel class for executing server-side Gemini templates.
getTemplateImagenModel()
getTemplateImagenModel(
ai,requestOptions?):TemplateImagenModel
Defined in: ai/lib/index.ts:191
Beta
Returns a TemplateImagenModel class for executing server-side Imagen templates.