Análisis del modelo GLM 5.2 para desarrollo

A día de hoy creo que todos estamos de acuerdo en que la Inteligencia artificial ha venido para quedarse, así que es relativamente importante ser capaces de identificar cual el el mejor modelo y los mejores procesos para nuestros usos. 

En mi caso es principalmente el desarrollo en C# con backend.

1 - Introducción a GLM 5.2

Recientemente ha salido un modelo de lenguaje nuevo GLM 5.2 creado por la empresa china zai, completamente open source, disponible en huggingface, el cual a primera vísta tiene un rendimiento brutal, no solo comparado con otros modelos open soruce, donde es el mejor, sino comparado con modelos de pago

Fuente: https://docs.z.ai/guides/llm/glm-5.2

Ojo un modelo open source, no significa que lo vayas a correr en tu casa, con un ordenador cualquiera o u mac mini, hay algunos que sí, de hecho, como hemos dicho está disponible para descargar, pero correrlo es un tema. 

Para correr el modelo GLM-5.2 completo y sin cuantizar es prácticamente imposible porque necesitas un hardware de locos.

• 753 Mil Millones de parámetros x 2 Bytes = 1.506 GB (1.5 Terabytes) de VRAM netos solo para cargar los pesos del modelo.
• Si sumamos el contexto y el procesamiento (un 20% de margen obligatorio), la cifra real para que funcione de forma estable sube a unos 1.8 Terabytes de VRAM.

O en otras palabras, entre 200-300k en tarjetas gráficas, asi que nos olvidamos

Lo que si que se puede hacer es para correr el modelo en local es cuantizar el modelo, lo que hace bajar el rendimiento de dicho modelo al 75-85% del rendimiento original, pero ahí ya lo puedes correr en casa relativamente bien, aunque vaya lento.

Así que en nuestro caso para correrlo lo tenemos que hacer a través de alguna plataforma, y lo vamos a utilizar online, igual que haríamos con los de openIA o con los de Anthropic, por lo que viene bien comparar el precio, en el caso de GLM 5.2 es 1M de tokens, en 

Como vemos el modelo GLM 5.2 es 5 veces más barato que GPT-5.5 o incluso que opus 4.8

para probarlo en local neceistas una cuenta en zai, coger la api token y ponerla en cursor, opencode, o incluso en claude code si quieres.

2 - Evaluacion de GLM 5.2 para desarrollo

LA forma en la que yo evaluo es simulando un par de tareas reales que podrían preguntar en una empresa, ambas sobre el repositorio de Distribt que está en github, el cual es un mono-repo con un sistema distribuido.

Y luego los criterios de evaluación y el resultado los documento en mi repo de las evaluaciones de código 

Si quieres ver un mayor razonamiento, explicaciones de porqué se hacen cosas como se hacen, etc, está todo disponible en un directo en youtube, el test es un par de prompts, pero hay que asegurarse de antes que no utiliza memoria o información de otros sitios, etc. 

Pero vamos a lo importante, en qué consisten las pruebas. 

2.1 - GLM 5.2 para hacer code reviews

La primera prueba es una code review, donde hay una serie de problemas en el código y trampas puestas. Aquí evaluamos como hace el modelo para evaluar. El prompt que enviamos es muy sencillo:

You are reviewing this code change (last commit) before it merges. Identify every finding you would report on.

You are reviewing this code change (last commit) before it merges. Identify every finding you would report on.
For each finding do a report like the following: 
--
    - Severity (Blocking, important, minor)
    - category
    - filename and extension
    - location
    - one line root cause
    - impact
--
Do not fix anything, this is a review.
return in markdown format.

Y por supuesto tenemos una lista con lo que esperamos que responda, el código base sobre el que corremos el modelo, no está puesto a ir a pillar, es la simulación de una implementación, y la PR la puedes encontrar aqui. 

Después de correr el promtp, esta es la salida del LLM:

Severity: Blocking
Category: Concurrency / thread-safety
File: UpdateProductPrice.cs
Location: Line 17 (static readonly Dictionary<int,decimal> LastPublishedPrice), accessed lines 33–37
Root cause: A static Dictionary<,> (non-thread-safe) is read and written from a scoped instance with no synchronization.
Impact: Under concurrent PUT /updateprice/{id} requests the dictionary can corrupt, throw, or infinite-loop; behavior is undefined and the API can hang or 500.
--

Severity: Blocking
Category: Memory leak / unbounded state
File: UpdateProductPrice.cs
Location: Line 17 (static dictionary never pruned)
Root cause: Entries are added to LastPublishedPrice per product id and never evicted.
Impact: Unbounded growth over the process lifetime; long-running API instances leak memory proportional to distinct products ever repriced.
--

Severity: Blocking
Category: Logic / correctness
File: UpdateProductPrice.cs
Location: Lines 33–37 (cache compares/stores request.Price, not finalPrice)
Root cause: The dedup cache keys on id and stores the input request.Price, but the published value is finalPrice (which depends on DiscountPercentage).
Impact: A second call with the same Price but a different DiscountPercentage is silently dropped — neither published nor sent to warehouse — producing a wrong persisted price.
--

Severity: Blocking
Category: Reliability / error handling
File: UpdateProductPrice.cs
Location: Lines 37–48 (cache updated before warehouse call; catch {} swallows error)
Root cause: LastPublishedPrice[id] = request.Price is set before the warehouse call, and any exception is swallowed.
Impact: A transient warehouse failure permanently marks the price as done; subsequent retries with the same price are skipped silently, so the warehouse never receives the update.
--

Severity: Blocking
Category: Observability / error handling
File: UpdateProductPrice.cs
Location: Lines 41–48 (catch { // pricing backend is best-effort })
Root cause: Empty catch swallows every exception with no logging, metric, or event.
Impact: Warehouse failures are invisible to operators; no way to detect, alert on, or debug pricing-backend outages.
--

Severity: Important
Category: API contract / misleading return
File: UpdateProductPrice.cs and ProductController.cs
Location: Execute returns true unconditionally (line 50); controller wraps it as ResultDto<bool> (line 49)
Root cause: The bool result is hardcoded true even when the warehouse call threw.
Impact: Callers cannot distinguish success from failure; the API always answers 200 OK, hiding real errors from clients.
--

Severity: Important
Category: Input validation
File: ProductDto.cs / UpdateProductPrice.cs
Location: UpdateProductPriceRequest(decimal Price, int DiscountPercentage) (line 13) and lines 27–28 of UpdateProductPrice.cs
Root cause: No validation on Price (≥0) or DiscountPercentage (0–100).
Impact: DiscountPercentage > 100 produces a negative finalPrice; DiscountPercentage < 0 increases the price; negative Price flows straight through to the warehouse and published event.
--

Severity: Important
Category: Numeric precision
File: UpdateProductPrice.cs
Location: Lines 30–31 (Math.Round((double)finalPrice, 2) then cast back to decimal)
Root cause: Rounding is done in double instead of decimal, then cast back.
Impact: Decimal→double→decimal is lossy; for some prices this introduces 1-cent discrepancies in a financial value. Should be Math.Round(finalPrice, 2).
--

Severity: Important
Category: Cancellation propagation
File: ProductPriceChangedHandler.cs
Location: Line 13 (await readStore.UpdateProductPrice(message.Content.ProductId, message.Content.Price);)
Root cause: The handler has a cancellationToken but does not forward it to readStore.UpdateProductPrice.
Impact: Store call cannot be cancelled on shutdown; inconsistent with ProductCreatedHandler/ProductUpdatedHandler which do forward the token.
--

Severity: Important
Category: Test isolation / shared state
File: UpdateProductPriceTests.cs
Location: Whole class — instantiates UpdateProductPrice which mutates the static LastPublishedPrice
Root cause: Tests don't reset the static cache; each test run depends on prior state.
Impact: Re-running the same test in one process (retry, [Theory], runner reuse) makes ModifySalesPrice not be called and Times.Once fails; the choice of id=2 in the second test is a workaround rather than a fix.
--

Severity: Important
Category: Test quality / coverage
File: UpdateProductPriceTests.cs
Location: Lines 22–23 (warehouse.Verify(w => w.ModifySalesPrice(It.IsAny<int>(), It.IsAny<decimal>()), Times.Once))
Root cause: Assertion uses It.IsAny<decimal>() and never checks Publish was called.
Impact: Test passes even if the discount math is completely broken; the discount calculation (the core of the feature) is not verified at all.
--

Severity: Important
Category: Test encodes buggy behaviour
File: UpdateProductPriceTests.cs
Location: Lines 26–37 (Execute_WhenWarehouseUpdateFails_StillReportsSuccess)
Root cause: Test asserts that swallowed exceptions still return true.
Impact: Locks in the silent-failure behaviour, making it harder to fix the underlying bug later without breaking the test.
--

Severity: Minor
Category: Convention consistency
File: ProductPriceChangedHandler.cs
Location: Lines 6–9 (primary constructor with readStore, integrationMessagePublisher)
Root cause: Uses primary-constructor parameters as fields; sibling handlers (ProductCreatedHandler, ProductUpdatedHandler) use explicit _-prefixed private fields with a constructor body.
Impact: Inconsistent style within the same folder; minor readability/consistency cost.
--

Severity: Minor
Category: Project references / cleanliness
File: Distribt.Tests.Services.Products.csproj
Location: Lines 23–25 (ProjectReference to Api.Read and Api.Write)
Root cause: Tests only use types from BusinessLogic; Api.Read is unused and Api.Write is referenced only transitively.
Impact: Builds unnecessarily slower; deploys test dependencies that aren't needed. Sibling test project (Orders.BusinessLogicTests) references the BusinessLogic project directly.
--

Severity: Minor
Category: Dependency versions / packaging
File: Distribt.Tests.Services.Products.csproj
Location: Lines 9–14
Root cause: Uses newer package versions than sibling test projects (Microsoft.NET.Test.Sdk 17.14.1 vs 16.11.0, xunit 2.9.2 vs 2.4.1, coverlet.collector 6.0.4 vs 3.1.0) and omits the PrivateAssets/IncludeAssets annotations used elsewhere.
Impact: Version drift across the solution; pulls analyzers/runtime assets into the test output that other projects suppress.
--

Severity: Minor
Category: Code hygiene
File: UpdateProductPrice.cs, ProductPriceChangedHandler.cs, ProductDto.cs, UpdateProductPriceTests.cs, Distribt.Tests.Services.Products.csproj, ProductController.cs
Location: End of each file (diff shows \ No newline at end of file)
Root cause: Files written without a trailing newline.
Impact: Noisy diffs later, fails some linters; minor.
--

Severity: Minor
Category: Misleading comment
File: UpdateProductPrice.cs
Location: Line 16 (// cache the last price we pushed per product so repeated calls don't re-publish ProductPriceChanged)
Root cause: Comment claims dedup is only for publishing, but the early return also skips the warehouse call.
Impact: Maintainers misunderstand the cache's effect; the warehouse update is also bypassed on a cache hit, not just the publish.
Summary: The most serious issues centre on the static LastPublishedPrice cache (thread-safety, memory leak, wrong key, premature caching combined with swallowed exceptions). Combined with the always-true return and the empty catch, the endpoint can silently lose price updates while reporting success to the client. The tests then lock in that broken behaviour. I'd recommend fixing the cache (or removing it), the error handling, and the precision/validation issues before merge.

Y finalmente el resultado es de 80 puntos, donde el máximo de puntos es 100:

NOTA: si quieres ver el análisis de cada uno de los puntos de forma extendida, dale al vídeo, es mucho para un post. 

2.2 - GLM 5.2 para desarrollar funcionalidades

La segunda tarea consiste en el desarrollo de una funcionalidad, en el código ya hay una funcionalidad similar desarrollada, lo que se espera es que genere un endpoint para guardar en una base de datos, genere un evento de dominio y escuchemos al evento de dominio en un consumer el cual propaga la información si es necesario, etc. 

El anterior punto tenía trampas que si se caía en ellas restan puntos, este tiene puntos extra, que si se hacen, suman puntos.

Este es el prompt: 

Add the ability to cancel an Order end to end.
- An order that has already been dispatched/delivered cannot be cancelled, return a sensible error.

Y este es el resultado:

What was added
Domain layer

Events/OrderEvents.cs — new OrderCancelled(string Reason) event + Cancelled status in the OrderStatus enum.
Aggregates/OrderDetails.cs — implemented IApply<OrderCancelled> (sets Status = Cancelled, records the change).
Aggregates/MongoMapping.cs — registered OrderCancelled in the BSON class maps.
Application layer

Services/OrderCancelledService.cs (new) — loads the aggregate, validates, applies the event, persists, then publishes a domain message. Returns Result<OrderCancelledResponse>:
order not found → Result.NotFound → 404
already Dispatched/Completed → Result.Conflict → 409 with message "Order {id} cannot be cancelled because it has already been dispatched/delivered"
otherwise applies OrderCancelled, saves to the event store, publishes OrderCancelledResponse to the order exchange.
API

Controllers/OrderController.cs — injected IOrderCancelledService, added PUT /order/cancel?orderId=...&reason=... (returns 202 Accepted / 404 / 409).
Program.cs — registered IOrderCancelledService.
DTO

OrderRequest.cs — added OrderCancelledResponse(Guid OrderId, string Reason) (used as both the API payload and the bus message; placed in the Dto project so the Consumer can deserialize it).
Consumer

Handler/OrderCancelledHandler.cs (new) — IDomainMessageHandler<OrderCancelledResponse>, auto-registered via the existing assembly scan, mirroring OrderCreatedHandler (stub for the cancellation saga — refund/release inventory).
End-to-end flow
PUT /order/cancel → OrderCancelledService (validates state, returns 409 if dispatched/delivered, 404 if missing) → OrderDetails.Apply(OrderCancelled) → OrderRepository.Save (Mongo event store, optimistic concurrency) → IDomainMessagePublisher.Publish (RabbitMQ order.exchange) → OrderCancelledHandler consumes DomainMessage<OrderCancelledResponse>.

Verification
Distribt.Services.Orders (API) builds: 0 errors.
Distribt.Services.Orders.Consumer builds: 0 errors.
Existing Orders BusinessLogic tests: 4/4 passed.
A note: I followed the existing convention of putting state-transition validation in the service (the aggregate's Apply methods have no guards here). I used a dedicated OrderCancelledResponse payload rather than reusing OrderResponse because the message dispatcher (HandleMessage.cs:28-41) invokes only the first matching handler per message type, so sharing DomainMessage<OrderResponse> between OrderCreatedHandler and a cancellation handler would cause non-deterministic routing.

Con un resultado de 70 puntos sobre un maximo de 140 incluyendo bonus, y de 100 sin los bonus.

Igual que antes, si quieres ver el análisis de cada punto de forma individual te invito a que vayas al video.

3 - Conclusión

El resultado final a dia de hoy (finales junio 2026) es que es un modelo MUY top a un precio muy bajo. el resultado a dia de hoy, sin tener los de antrhopic evaluados es que es el mejor, mucho mejor que GPT 5.5, y por una fracción del precio.

Sobre qué modelo utilizar, eso te lo dejo ya a ti para decidir.