Analysis of the GLM 5.2 model for development

As of today, I think we all agree that Artificial Intelligence is here to stay, so it is relatively important to be able to identify which is the best model and the best processes for our use cases. 

In my case, it is mainly C# backend development.

1 - Introduction to GLM 5.2

Recently, a new language model called GLM 5.2 was released by the Chinese company Z.ai, completely open source, available on Hugging Face. At first glance, it has brutal performance, not only compared with other open source models, where it is the best, but also compared with paid models.

Source: https://docs.z.ai/guides/llm/glm-5.2

Careful, an open source model does not mean you are going to run it at home with any random computer or a Mac mini. Some of them, yes. In fact, as we said, it is available to download, but running it is another matter. 

To run the full, unquantized GLM-5.2 model is practically impossible because you need insane hardware.

• 753 billion parameters x 2 bytes = 1,506 GB (1.5 terabytes) of VRAM net just to load the model weights.
• If we add context and processing (a mandatory 20% margin), the real figure for it to run stably goes up to around 1.8 terabytes of VRAM.

Or in other words, between 200k and 300k in graphics cards, so we can forget about that.

What can be done to run the model locally is to quantize the model, which lowers that model's performance to 75 to 85% of the original performance, but then you can run it at home relatively well, even if it is slow.

So in our case, to run it we have to do it through some platform, and we are going to use it online, just like we would with OpenAI or Anthropic models, so it is useful to compare the price. In the case of GLM 5.2, this is per 1M tokens:

As we can see, the GLM 5.2 model is 5 times cheaper than GPT-5.5 or even Opus 4.8.

To test it locally, you need an account in Z.ai, get the API token and put it in Cursor, OpenCode, or even in Claude Code if you want.

2 - Evaluation of GLM 5.2 for development

The way I evaluate it is by simulating a couple of real tasks that could be asked in a company, both on the Distribt repository that is on GitHub, which is a mono-repo with a distributed system.

And then I document the evaluation criteria and the result in my code evaluation repo 

If you want to see more reasoning, explanations of why things are done the way they are done, etc., everything is available in a livestream on YouTube. The test is a couple of prompts, but you have to make sure beforehand that it is not using memory or information from other places, etc. 

But let's get to the important part, what the tests consist of. 

2.1 - GLM 5.2 for code reviews

The first test is a code review, where there are a series of problems in the code and some traps placed. Here we evaluate how the model performs when reviewing. The prompt we send is very simple:

You are reviewing this code change (last commit) before it merges. Identify every finding you would report on.

You are reviewing this code change (last commit) before it merges. Identify every finding you would report on.
For each finding do a report like the following: 
--
    - Severity (Blocking, important, minor)
    - category
    - filename and extension
    - location
    - one line root cause
    - impact
--
Do not fix anything, this is a review.
return in markdown format.

And of course we have a list of what we expect it to answer. The base code on which we run the model is not made to catch it out, it is the simulation of an implementation, and you can find the PR here. 

After running the prompt, this is the LLM output:

Severity: Blocking
Category: Concurrency / thread-safety
File: UpdateProductPrice.cs
Location: Line 17 (static readonly Dictionary<int,decimal> LastPublishedPrice), accessed lines 33–37
Root cause: A static Dictionary<,> (non-thread-safe) is read and written from a scoped instance with no synchronization.
Impact: Under concurrent PUT /updateprice/{id} requests the dictionary can corrupt, throw, or infinite-loop; behavior is undefined and the API can hang or 500.
--

Severity: Blocking
Category: Memory leak / unbounded state
File: UpdateProductPrice.cs
Location: Line 17 (static dictionary never pruned)
Root cause: Entries are added to LastPublishedPrice per product id and never evicted.
Impact: Unbounded growth over the process lifetime; long-running API instances leak memory proportional to distinct products ever repriced.
--

Severity: Blocking
Category: Logic / correctness
File: UpdateProductPrice.cs
Location: Lines 33–37 (cache compares/stores request.Price, not finalPrice)
Root cause: The dedup cache keys on id and stores the input request.Price, but the published value is finalPrice (which depends on DiscountPercentage).
Impact: A second call with the same Price but a different DiscountPercentage is silently dropped — neither published nor sent to warehouse — producing a wrong persisted price.
--

Severity: Blocking
Category: Reliability / error handling
File: UpdateProductPrice.cs
Location: Lines 37–48 (cache updated before warehouse call; catch {} swallows error)
Root cause: LastPublishedPrice[id] = request.Price is set before the warehouse call, and any exception is swallowed.
Impact: A transient warehouse failure permanently marks the price as done; subsequent retries with the same price are skipped silently, so the warehouse never receives the update.
--

Severity: Blocking
Category: Observability / error handling
File: UpdateProductPrice.cs
Location: Lines 41–48 (catch { // pricing backend is best-effort })
Root cause: Empty catch swallows every exception with no logging, metric, or event.
Impact: Warehouse failures are invisible to operators; no way to detect, alert on, or debug pricing-backend outages.
--

Severity: Important
Category: API contract / misleading return
File: UpdateProductPrice.cs and ProductController.cs
Location: Execute returns true unconditionally (line 50); controller wraps it as ResultDto<bool> (line 49)
Root cause: The bool result is hardcoded true even when the warehouse call threw.
Impact: Callers cannot distinguish success from failure; the API always answers 200 OK, hiding real errors from clients.
--

Severity: Important
Category: Input validation
File: ProductDto.cs / UpdateProductPrice.cs
Location: UpdateProductPriceRequest(decimal Price, int DiscountPercentage) (line 13) and lines 27–28 of UpdateProductPrice.cs
Root cause: No validation on Price (≥0) or DiscountPercentage (0–100).
Impact: DiscountPercentage > 100 produces a negative finalPrice; DiscountPercentage < 0 increases the price; negative Price flows straight through to the warehouse and published event.
--

Severity: Important
Category: Numeric precision
File: UpdateProductPrice.cs
Location: Lines 30–31 (Math.Round((double)finalPrice, 2) then cast back to decimal)
Root cause: Rounding is done in double instead of decimal, then cast back.
Impact: Decimal→double→decimal is lossy; for some prices this introduces 1-cent discrepancies in a financial value. Should be Math.Round(finalPrice, 2).
--

Severity: Important
Category: Cancellation propagation
File: ProductPriceChangedHandler.cs
Location: Line 13 (await readStore.UpdateProductPrice(message.Content.ProductId, message.Content.Price);)
Root cause: The handler has a cancellationToken but does not forward it to readStore.UpdateProductPrice.
Impact: Store call cannot be cancelled on shutdown; inconsistent with ProductCreatedHandler/ProductUpdatedHandler which do forward the token.
--

Severity: Important
Category: Test isolation / shared state
File: UpdateProductPriceTests.cs
Location: Whole class — instantiates UpdateProductPrice which mutates the static LastPublishedPrice
Root cause: Tests don't reset the static cache; each test run depends on prior state.
Impact: Re-running the same test in one process (retry, [Theory], runner reuse) makes ModifySalesPrice not be called and Times.Once fails; the choice of id=2 in the second test is a workaround rather than a fix.
--

Severity: Important
Category: Test quality / coverage
File: UpdateProductPriceTests.cs
Location: Lines 22–23 (warehouse.Verify(w => w.ModifySalesPrice(It.IsAny<int>(), It.IsAny<decimal>()), Times.Once))
Root cause: Assertion uses It.IsAny<decimal>() and never checks Publish was called.
Impact: Test passes even if the discount math is completely broken; the discount calculation (the core of the feature) is not verified at all.
--

Severity: Important
Category: Test encodes buggy behaviour
File: UpdateProductPriceTests.cs
Location: Lines 26–37 (Execute_WhenWarehouseUpdateFails_StillReportsSuccess)
Root cause: Test asserts that swallowed exceptions still return true.
Impact: Locks in the silent-failure behaviour, making it harder to fix the underlying bug later without breaking the test.
--

Severity: Minor
Category: Convention consistency
File: ProductPriceChangedHandler.cs
Location: Lines 6–9 (primary constructor with readStore, integrationMessagePublisher)
Root cause: Uses primary-constructor parameters as fields; sibling handlers (ProductCreatedHandler, ProductUpdatedHandler) use explicit _-prefixed private fields with a constructor body.
Impact: Inconsistent style within the same folder; minor readability/consistency cost.
--

Severity: Minor
Category: Project references / cleanliness
File: Distribt.Tests.Services.Products.csproj
Location: Lines 23–25 (ProjectReference to Api.Read and Api.Write)
Root cause: Tests only use types from BusinessLogic; Api.Read is unused and Api.Write is referenced only transitively.
Impact: Builds unnecessarily slower; deploys test dependencies that aren't needed. Sibling test project (Orders.BusinessLogicTests) references the BusinessLogic project directly.
--

Severity: Minor
Category: Dependency versions / packaging
File: Distribt.Tests.Services.Products.csproj
Location: Lines 9–14
Root cause: Uses newer package versions than sibling test projects (Microsoft.NET.Test.Sdk 17.14.1 vs 16.11.0, xunit 2.9.2 vs 2.4.1, coverlet.collector 6.0.4 vs 3.1.0) and omits the PrivateAssets/IncludeAssets annotations used elsewhere.
Impact: Version drift across the solution; pulls analyzers/runtime assets into the test output that other projects suppress.
--

Severity: Minor
Category: Code hygiene
File: UpdateProductPrice.cs, ProductPriceChangedHandler.cs, ProductDto.cs, UpdateProductPriceTests.cs, Distribt.Tests.Services.Products.csproj, ProductController.cs
Location: End of each file (diff shows \ No newline at end of file)
Root cause: Files written without a trailing newline.
Impact: Noisy diffs later, fails some linters; minor.
--

Severity: Minor
Category: Misleading comment
File: UpdateProductPrice.cs
Location: Line 16 (// cache the last price we pushed per product so repeated calls don't re-publish ProductPriceChanged)
Root cause: Comment claims dedup is only for publishing, but the early return also skips the warehouse call.
Impact: Maintainers misunderstand the cache's effect; the warehouse update is also bypassed on a cache hit, not just the publish.
Summary: The most serious issues centre on the static LastPublishedPrice cache (thread-safety, memory leak, wrong key, premature caching combined with swallowed exceptions). Combined with the always-true return and the empty catch, the endpoint can silently lose price updates while reporting success to the client. The tests then lock in that broken behaviour. I'd recommend fixing the cache (or removing it), the error handling, and the precision/validation issues before merge.

And finally the result is 80 points, where the maximum score is 100:

NOTE: if you want to see the analysis of each point in detail, watch the video, it is too much for a post. 

2.2 - GLM 5.2 for developing features

The second task consists of developing a feature. There is already a similar feature developed in the code, and what is expected is that it generates an endpoint to save to a database, generates a domain event, and that we listen to the domain event in a consumer which propagates the information if necessary, etc. 

The previous point had traps that subtract points if it falls into them. This one has extra points, which add points if they are done.

This is the prompt: 

Add the ability to cancel an Order end to end.
- An order that has already been dispatched/delivered cannot be cancelled, return a sensible error.

And this is the result:

What was added
Domain layer

Events/OrderEvents.cs — new OrderCancelled(string Reason) event + Cancelled status in the OrderStatus enum.
Aggregates/OrderDetails.cs — implemented IApply<OrderCancelled> (sets Status = Cancelled, records the change).
Aggregates/MongoMapping.cs — registered OrderCancelled in the BSON class maps.
Application layer

Services/OrderCancelledService.cs (new) — loads the aggregate, validates, applies the event, persists, then publishes a domain message. Returns Result<OrderCancelledResponse>:
order not found → Result.NotFound → 404
already Dispatched/Completed → Result.Conflict → 409 with message "Order {id} cannot be cancelled because it has already been dispatched/delivered"
otherwise applies OrderCancelled, saves to the event store, publishes OrderCancelledResponse to the order exchange.
API

Controllers/OrderController.cs — injected IOrderCancelledService, added PUT /order/cancel?orderId=...&reason=... (returns 202 Accepted / 404 / 409).
Program.cs — registered IOrderCancelledService.
DTO

OrderRequest.cs — added OrderCancelledResponse(Guid OrderId, string Reason) (used as both the API payload and the bus message; placed in the Dto project so the Consumer can deserialize it).
Consumer

Handler/OrderCancelledHandler.cs (new) — IDomainMessageHandler<OrderCancelledResponse>, auto-registered via the existing assembly scan, mirroring OrderCreatedHandler (stub for the cancellation saga — refund/release inventory).
End-to-end flow
PUT /order/cancel → OrderCancelledService (validates state, returns 409 if dispatched/delivered, 404 if missing) → OrderDetails.Apply(OrderCancelled) → OrderRepository.Save (Mongo event store, optimistic concurrency) → IDomainMessagePublisher.Publish (RabbitMQ order.exchange) → OrderCancelledHandler consumes DomainMessage<OrderCancelledResponse>.

Verification
Distribt.Services.Orders (API) builds: 0 errors.
Distribt.Services.Orders.Consumer builds: 0 errors.
Existing Orders BusinessLogic tests: 4/4 passed.
A note: I followed the existing convention of putting state-transition validation in the service (the aggregate's Apply methods have no guards here). I used a dedicated OrderCancelledResponse payload rather than reusing OrderResponse because the message dispatcher (HandleMessage.cs:28-41) invokes only the first matching handler per message type, so sharing DomainMessage<OrderResponse> between OrderCreatedHandler and a cancellation handler would cause non-deterministic routing.

With a result of 70 points out of a maximum of 140 including bonuses, and out of 100 without the bonuses.

Same as before, if you want to see the analysis of each point individually, I invite you to go to the video.

3 - Conclusion

The final result as of today (late June 2026) is that it is a VERY top model at a very low price. The result today, without having evaluated Anthropic's models, is that it is the best, much better than GPT 5.5, and for a fraction of the price.

As for which model to use, I will leave that up to you to decide.