TailoredApps.Shared.Mediator.ML¶
Description¶
This library integrates ML.NET image classification with the mediator pipeline. It provides ready-made commands (ClassifyImage, TrainImageClassificationModel) and their handlers, making image classification a first-class citizen in CQRS architecture.
Under the hood, ImageClassificationService uses ML.NET with a PredictionEnginePool for efficient concurrent inference. The library also supports in-app model training — you can train a new model by providing a labeled image dataset.
Installation¶
DI registration¶
// Program.cs
using Microsoft.Extensions.ML;
using TailoredApps.Shared.Mediator.ImageClassification.Infrastructure;
using TailoredApps.Shared.Mediator.ImageClassification.Interfaces.Domain.Models;
// Mediator: the handlers of both commands live in the library assembly, so scan it as well
builder.Services.AddMediator(cfg =>
{
cfg.RegisterServicesFromAssembly(typeof(Program).Assembly);
cfg.RegisterServicesFromAssembly(typeof(AddPredictionEngineExtension).Assembly);
});
// Library services + the ML.NET PredictionEnginePool with the model
builder.Services.AddPredictionEngine(config =>
{
config.RegisterMachineLearningModel<InMemoryImageData, ImagePredictionScore>(pool =>
{
pool.FromFile(builder.Configuration[ImageClassificationOptions.ModelFilePathConfig]!);
});
});
AddPredictionEngine registers IImageClassificationService, IModelInfoService and IModelHelper (scoped) and binds ImageClassificationOptions. It does not register the mediator handlers - that is what scanning the library assembly is for. RegisterMachineLearningModel adds the ML.NET PredictionEnginePool; where the model is loaded from is decided by the pool builder (FromFile above). Register the model without a name: predictions use the default model of the pool.
The types live in the namespace TailoredApps.Shared.Mediator.ImageClassification, not under the package name.
appsettings.json configuration¶
{
"ImageClassification": {
"ModelFilePath": "/var/app/models/current.zip",
"TrainingRoot": "/var/app/training-sets",
"ModelsRoot": "/var/app/models",
"MaxImageBytes": 10485760
}
}
| Key | Meaning | Default |
|---|---|---|
ModelFilePath |
Model file described by IImageClassificationService.GetModelInfo() (name, checksum, version, labels); the registration above also loads the model from it |
none |
TrainingRoot |
Directory that training sets must be inside; required for training | none |
ModelsRoot |
Directory that trained models are written inside; required for training | none |
MaxImageBytes |
Largest image accepted by ClassifyImage, in bytes; 0 turns the limit off |
10485760 (10 MB) |
The section is required: without it the options throw InvalidOperationException when they are first used.
Usage example¶
Image classification¶
public class ImageAnalysisController : ControllerBase
{
private readonly IMediator _mediator;
public ImageAnalysisController(IMediator mediator) => _mediator = mediator;
[HttpPost("classify")]
public async Task<IActionResult> ClassifyImage(IFormFile imageFile)
{
using var ms = new MemoryStream();
await imageFile.CopyToAsync(ms);
var result = await _mediator.Send(new ClassifyImage
{
FileByteArray = ms.ToArray(),
FileName = imageFile.FileName
});
var prediction = result.ImagePrediction!; // the handler always sets it
return Ok(new
{
prediction.FileName,
prediction.PredictedLabel,
prediction.PredictedScore,
Confidence = $"{prediction.PredictedScore:P2}"
});
}
}
Model training¶
var trainingResult = await _mediator.Send(new TrainImageClassificationModel
{
Source = "pets", // directory inside ImageClassification:TrainingRoot, one subfolder per class
ModelDestFolderPath = "pets.zip" // model file inside ImageClassification:ModelsRoot
});
Console.WriteLine($"Model {trainingResult.ModelPath}, version {trainingResult.ModelVersion}");
Console.WriteLine($"Labels: {string.Join(", ", trainingResult.Labels)}");
Console.WriteLine(trainingResult.ModelInfo); // evaluation metrics as text
Training reads the files with the extension .jpg or .png from the source directory and its subfolders; the name of the folder a file is in becomes its label.
API Reference¶
| Type | Kind | Description |
|---|---|---|
ClassifyImage |
Command | Image data to classify: FileByteArray, FileName |
ClassifyImageResponse |
Class | Result: ImagePrediction |
TrainImageClassificationModel |
Command | Training parameters: Source, ModelDestFolderPath |
TrainImageClassificationModelResponse |
Class | Training result: ModelPath, ModelVersion, ModelInfo (evaluation metrics as text), Labels |
IImageClassificationService |
Interface | Predict(byte[], fileName), Train(images, folder, dest), GetModelInfo() |
IPredictionEnginePoolAdapter<TData, TPred> |
Interface | Abstraction over the ML.NET PredictionEnginePool |
ModelInfo |
Class | Model metadata: name, checksum, version, labels |
ImagePrediction |
Class | Prediction result: PredictedLabel, PredictedScore, FileName; Scores and ModelInfo are not filled by ClassifyImage |
InMemoryImageData |
Class | Model input data: the image as byte[] |
ImagePredictionScore |
Class | Raw model output: Score (per class), PredictedLabel |
ImageClassificationOptions |
Class | Options bound from the ImageClassification section: ModelFilePath, TrainingRoot, ModelsRoot, MaxImageBytes |
AddPredictionEngineExtension.AddPredictionEngine |
Ext. method | Registers IImageClassificationService, IModelInfoService, IModelHelper and the options in DI (not the mediator handlers) |
PredictionEngineServiceConfiguration.RegisterMachineLearningModel<D, R> |
Method | Adds the ML.NET PredictionEnginePool<D, R> and its adapter; called inside AddPredictionEngine |
🔒 Security¶
- Training paths are confined to configured directories.
TrainImageClassificationModel.Sourcemust resolve insideImageClassification:TrainingRootandModelDestFolderPathinsideImageClassification:ModelsRoot(relative paths are resolved against those roots). A path outside its root throwsArgumentException; missing root configuration throwsInvalidOperationException. Without this a request could recursively read any folder on the host and overwrite any file. - Images are validated before the decoder runs.
ClassifyImagerejects (ArgumentException) empty content, files larger thanImageClassification:MaxImageBytes(default 10 MB) and content without a JPEG/PNG signature.
{
"ImageClassification": {
"ModelFilePath": "/var/app/models/current.zip",
"TrainingRoot": "/var/app/training-sets",
"ModelsRoot": "/var/app/models",
"MaxImageBytes": 10485760
}
}
🤖 AI Agent Prompt¶
## TailoredApps.Shared.Mediator.ML - instructions for an AI agent
You are using TailoredApps.Shared.Mediator.ML to classify images through ML.NET and the mediator.
### Registration
```csharp
// using Microsoft.Extensions.ML;
// using TailoredApps.Shared.Mediator.ImageClassification.Infrastructure;
// using TailoredApps.Shared.Mediator.ImageClassification.Interfaces.Domain.Models;
builder.Services.AddMediator(cfg =>
{
cfg.RegisterServicesFromAssembly(typeof(Program).Assembly);
cfg.RegisterServicesFromAssembly(typeof(AddPredictionEngineExtension).Assembly); // handlers of the library
});
builder.Services.AddPredictionEngine(config =>
config.RegisterMachineLearningModel<InMemoryImageData, ImagePredictionScore>(pool =>
pool.FromFile(builder.Configuration[ImageClassificationOptions.ModelFilePathConfig]!)));
```
### What gets configured
- IImageClassificationService, IModelInfoService, IModelHelper (scoped) and ImageClassificationOptions
- The ML.NET PredictionEnginePool<InMemoryImageData, ImagePredictionScore> with the model (default, unnamed)
- The handlers of ClassifyImage and TrainImageClassificationModel - only through the mediator scan of the library assembly
### Configuration (section "ImageClassification")
- ModelFilePath - the model file (used for the model metadata and by the registration above)
- TrainingRoot, ModelsRoot - directories training sets are read from / models are written to; required for training
- MaxImageBytes - image size limit for ClassifyImage, default 10485760 (10 MB), 0 = no limit
### Image classification
```csharp
var result = await _mediator.Send(new ClassifyImage
{
FileByteArray = imageBytes,
FileName = "photo.jpg"
});
// result.ImagePrediction.PredictedLabel — the predicted class
// result.ImagePrediction.PredictedScore — confidence (0-1)
```
### Model training
```csharp
var result = await _mediator.Send(new TrainImageClassificationModel
{
Source = "pets", // inside TrainingRoot; subfolders = class names
ModelDestFolderPath = "pets.zip" // model file inside ModelsRoot
});
// result.ModelPath, result.ModelVersion, result.Labels, result.ModelInfo (metrics as text)
```
### Rules
- The namespace is TailoredApps.Shared.Mediator.ImageClassification, not the package name
- AddMediator must scan the library assembly - AddPredictionEngine does not register the handlers
- The model must be a ZIP file (ML.NET format)
- The `ImageClassification` configuration section is required — a missing section throws `InvalidOperationException` when the options are first used
- The model version and labels are stored in the archive as separate `Version.txt` and `Labels.txt` entries
- Training folder: each subfolder = one class, folder name = label
- PredictionEnginePool is thread-safe — safe for concurrent use
- PredictedScore ∈ [0,1] — the closer to 1, the higher the model's confidence
- Configure ImageClassification:TrainingRoot and ImageClassification:ModelsRoot - training rejects paths outside those directories
- ClassifyImage accepts only JPEG/PNG up to MaxImageBytes (default 10 MB); larger or foreign files throw ArgumentException
- Expose the training command to administrators only - it starts a long TensorFlow job on the given data