Recommending a course, product, or service in a membership platform initially looks like a familiar “you may also like” problem. In practice, it requires more than estimating what someone is likely to click.
A member may already own the relevant package. A course may be unavailable at their location. A service may require an assessment that has not been completed. A popular resource may have little to do with the member’s current goal.
Our initial idea was to group similar members with K-means and recommend resources that other people in the same group had recently shown interest in. As the project developed, it became clear that clustering could help discover patterns but could not replace explicit business rules.
The approach became a hybrid: rule-based profiles, member segmentation, behavior-based candidate retrieval, semantic matching, and LLM-assisted interpretation and explanation.
This article describes that design evolution. The numerical weights and worked example are illustrative, not measured project results. Where I describe additional validation, failure handling, or evaluation, I frame it as guidance for making the approach more robust rather than claiming an implementation detail or business improvement that has not been documented.
1. Start with What the Recommendation Is Supposed to Achieve
Three questions shaped the design:
- What might this member be interested in now?
- Which resources are actually available and eligible for this member?
- What evidence supports the recommendation?
These questions overlap, but they are not interchangeable.
Consider a member who trains regularly, usually works with one coach, and has almost used up a session package. Those facts may support a timely conversation about scheduling or renewal. They do not automatically establish that the member wants to buy more sessions.
Similarly, a member who reports discomfort needs a different handling path from someone who has simply selected a general fitness goal. A recommendation model cannot infer training suitability from a symptom description alone.
This distinction is important in a fitness-related service: commercial relevance, personal interest, and professionally reviewed suitability are separate inputs. A high predicted conversion score must not override a required review or a resource restriction.
2. What K-Means Adds—and What It Does Not
We considered a feature vector covering activity, spending, benefits, and service continuity:
attendanceCount30d
attendanceCount180d
daysSinceLastAttendance
actualConsumptionAmount
remainingBenefitRate
mainCoachRate
planCompletionRate
homeworkCompletionRate
assessmentImprovementK-means groups members according to distances between their feature vectors. Its objective is to minimize within-cluster squared distances:
Here, (x_i) is a member’s feature vector, (z_i) is its assigned cluster, and (\mu_{z_i}) is the corresponding centroid.
The algorithm does not understand business labels such as “high renewal potential.” People must inspect the resulting distributions and decide whether a group has an operationally useful interpretation.
A cluster might have higher spending and lower recent attendance than the overall population. Calling it “higher-spending, less-active members” describes the observed features. Calling it “members likely to renew” adds a prediction that clustering alone has not established.
The scikit-learn clustering documentation describes the method and its assumptions.
Standardization is only the beginning
Spending amounts can dominate attendance counts unless features are scaled appropriately. Several other decisions also deserve attention:
- Long-tailed spending features may need transformation or clipping.
- Missing assessments should not silently become zero improvement.
- Overlapping activity windows can give activity more influence than intended.
- Improvement scores from different assessment types may not be comparable.
- New members may have too little history for a meaningful behavioral assignment.
These are feature-design decisions, not details that K-means fixes automatically.
For a maintained implementation, version the preprocessing with the clustering model, inspect cluster sizes and stability, and choose the number of clusters using both statistical checks and operational usefulness.
3. Separate Business Tags from Discovered Groups
Some labels already have explicit business definitions.
For example, a low-benefit rule could be:
totalBenefits > 0
remainingBenefits > 0
remainingBenefits / totalBenefits <= 0.20There is no need for a clustering algorithm to rediscover that threshold.
The calculation should apply to a consistent benefit type. A cash balance, a session count, and a course package cannot be added as if they were the same unit. A zero balance should also have a separate “exhausted” state rather than falling through the low-benefit rule.
A useful separation is:
| Type | Example | Interpretation |
|---|---|---|
| Explicit rule tag | Low remaining sessions | A documented rule was met |
| Descriptive metric | High main-coach share | Service is concentrated with one coach |
| Cluster description | Higher spending, lower recent activity | A pattern appears in a group |
| Predictive output | Estimated renewal probability | Requires a defined outcome and validation |
A high main-coach share does not prove psychological dependence. A rule called “renewal opportunity” does not establish a calibrated probability of renewal.
Clear names help operations teams understand what the system knows and what it is only suggesting.
Cluster IDs are not permanent business identities
After retraining, cluster numbers can be permuted or the underlying groups can change. Store enough context to interpret each assignment:
clusterId
clusterVersion
featureVersion
calculatedTimePersist the preprocessing and centroid definitions as well. “Cluster 3” has little meaning without the version that produced it.
4. Use Multiple Candidate Sources
Our starting point was to learn from the behavior of other members in the same group. This is useful, but it needs precise terminology.
Aggregating popular resources within a segment is cohort-based behavioral retrieval. It is related to the broader idea of learning from other users, but it is not automatically a full collaborative-filtering implementation.
Collaborative filtering models user–item interaction relationships—for example, through user similarity, item similarity, or learned interaction representations. If a system uses one of those methods, it should describe the actual method rather than treating segmentation itself as collaborative filtering. See Google’s introduction to collaborative filtering.
For this design, candidate sources can be described independently:
| Candidate source | Signal used |
|---|---|
| Cohort behavior | Recent interactions from similar member groups |
| Personal history | The member’s own past interests and usage |
| Content matching | Compatibility between stated needs and resource descriptions |
| Explicit operational rules | Resources relevant to a known state, where appropriate |
If collaborative filtering is available, it can contribute another candidate source or improve the history-based source. It does not need to replace the others.
This also avoids making clustering a prerequisite for every recommendation. A new member can receive eligible content-based candidates even without a reliable behavioral cluster.
5. Separate Periodic Preparation from Online Decisions
The architecture is easier to reason about when expensive profile preparation is separated from the request path.
Periodic work includes computing member metrics, assigning rule tags, refreshing clusters, aggregating cohort behavior, and updating resource metadata or embeddings.
At request time, the system reads available context, retrieves candidates from multiple sources, merges duplicates, checks eligibility, ranks the remaining resources, and optionally asks an LLM to assist with ordering or explanation.
Figure 1. A conceptual organization of the hybrid approach, including a recommended fallback path. Prepared features are read by the relevant online stages; individual read connections are omitted for clarity. The diagram does not specify actual refresh schedules or imply that every enhancement has already been deployed.
The important property is separation of responsibility. Clustering should not run synchronously for every request, and an LLM should not be responsible for deciding whether a booking slot still exists.
Some constraints can be pushed into retrieval to reduce unnecessary work. Final checks are still necessary because eligibility, capacity, and entitlement state may change between candidate generation and display.
6. Turn Behavior into Usable Signals
A first version can assign different strengths to interaction types. The following values are illustrative starting points, not a tuned production configuration:
| Event | Example weight |
|---|---|
| Resource view | 1 |
| Search-derived interaction with a resource | 2 |
| Detail view | 3 |
| Favorite | 5 |
| Add to cart | 8 |
| Booking | 10 |
| Purchase | 15 |
A search query without an associated resource belongs in a query-interest representation; it should not arbitrarily credit a product.
These events also have different meanings. A purchase is not proof of use, a booking is not attendance, and a view is not a clear preference. Repeated page refreshes should not outweigh a meaningful booking.
Deduplicate events, limit repeated contributions within a window, and decide whether successive events in the same journey accumulate or replace weaker signals.
Apply time decay
One interpretable decay function is:
where (H) is the half-life in the same time units as (\Delta t).
An event’s contribution becomes:
The half-life is a tuning choice. A temporary search interest and a long-running service relationship may need different treatment.
Aggregate within a group
A simple cohort score for resource (j) is:
Here, (U_{g,j}) is the defined comparison population for that resource, such as eligible active members in group (g), and (I_{u,j}) is the bounded, time-weighted interaction contribution.
The population definition matters. Including members who could never book a course can distort the result. A missing or empty comparison population requires a fallback, not division by zero.
A resource penetration measure can count distinct interacting members rather than total events. It is not a click-through rate unless its denominator is an appropriate exposure population.
Small groups also need minimum support or smoothing toward a broader baseline. Otherwise, one person’s purchase can dominate the recommendations of an entire tiny segment.
Do not treat every refund as dislike
A fixed negative refund weight is tempting but can be misleading. Refunds can result from availability, scheduling, duplicate payment, or service issues.
Keep refunds as a distinct signal with a reason where available. An explicit “not interested” response is different evidence from a refund caused by a canceled class.
7. Rank Only After Hard Constraints Are Satisfied
An illustrative first-version ranking formula is:
where:
- (C): cohort interest.
- (M): content match.
- (P): personal-history relevance.
- (F): freshness.
- (B): business priority.
Before applying this formula, bring the components onto comparable, documented scales. Otherwise, a cohort score ranging into the hundreds can overwhelm a content score between zero and one despite the apparent percentages.
The coefficients are starting hypotheses. They should be evaluated against a simpler baseline, not described as optimal values.
Business priority is a soft preference among eligible resources. It cannot compensate for a failed location check, missing prerequisite, unavailable capacity, or a reviewed restriction.
Benefits also depend on the action being recommended. Existing ownership may make a purchase recommendation redundant while making a booking recommendation more useful. “Already purchased” should therefore not be a blanket exclusion for every resource interaction.
A final result set may need diversity constraints to avoid recommending several nearly identical items. Again, this should not force the system to fill every slot when too few eligible resources exist.
8. Give the LLM a Bounded Job
LLMs are useful where the inputs are unstructured or the output needs natural language. In this project context, those jobs include interpreting member descriptions, normalizing resource descriptions, comparing candidate content, and explaining approved recommendations.
They should work within a contract set by the application.
Extract what the member actually said
Consider:
“I sit at a desk all day. My lower back often aches, and running has been uncomfortable lately.”
A careful extraction might be:
{
"reportedBodyPart": ["lower back"],
"reportedSymptoms": ["aching", "discomfort while running"],
"workPattern": ["prolonged sitting"],
"statedGoal": null,
"approvedIntensity": null
}The model should not silently turn this into “recovery phase,” “low-intensity training approved,” or “goal: return to running.” Those are additional interpretations or decisions.
Attach source text or evidence references to extracted fields where practical. Validate the output schema, and preserve uncertainty rather than filling every field.
When symptoms affect eligibility, the application should use the relevant reviewed assessment and defined human-review process. Text extraction is not a substitute for that review.
Normalize resource descriptions without inventing properties
Course names such as “Core Activation” or “Runner Mobility” do not establish intensity, prerequisites, or suitability for a particular condition.
An LLM can extract information explicitly present in approved course documentation. Missing properties should remain unknown and be routed for metadata review when they are required for recommendation.
Embeddings can help retrieve semantically related resources, but semantic similarity is not an eligibility decision.
Help with cold start
A new member may have little behavioral history but still provide a goal, location, schedule, and resource preferences.
Content matching and eligible popular resources can support an initial recommendation. An LLM is useful if free text needs interpretation; it is not required for cold start when the relevant fields are already structured.
Generate grounded explanations
A defensible explanation might be:
“This course matches your stated fitness goal, is available at your selected location, and has a session during your preferred time.”
That is different from claiming that a course will resolve discomfort or is clinically appropriate.
Reasons should reference information actually used by the system. Fluent text should not invent a personalized rationale after an unrelated commercial ranking decision.
For an implementation, a structured response can limit the model to approved candidate IDs and evidence references:
{
"rankedResources": [
{
"resourceId": "course_a",
"evidenceIds": ["goal_match", "location_match", "time_match"],
"reason": "Matches the stated goal, location, and available time."
}
]
}This is an example output contract, not a recovered production payload. The application must validate its identifiers and the supporting facts.
9. Walk Through One Recommendation
The following is a constructed example to illustrate the pipeline, not an anonymized record of a measured recommendation.
Assume a member has explicitly selected a general fitness goal, Store A, and weekday evenings. Any required restrictions are read from reviewed records rather than inferred by the LLM.
Multiple retrieval sources produce four courses:
| Resource | Decision | Reason |
|---|---|---|
| Course A | Keep | Matches the goal, location, time, and current eligibility checks |
| Course B | Exclude | Not available at the eligible location |
| Course C | Exclude | No available capacity |
| Course D | Hold | Required review has not been completed |
Figure 2. Popularity or semantic relevance does not override a failed eligibility check. With one eligible course remaining, no further relative ranking is needed in this simplified example.
The LLM may explain Course A using the supplied facts. It cannot bring Course B back because its description sounds relevant or claim that Course D is suitable despite the missing review.
Before returning the result, the service checks the ID and current availability again. Booking or purchase must perform its own transactional checks too: a recommendation does not reserve inventory.
If no course qualifies, the valid outcome may be an empty list, another search, or a request for staff assistance.
10. Design for Uncertain and Failed Model Responses
The original architectural principle was to keep deterministic business logic in Java. A practical extension is to make the optional LLM stage unable to break that contract.
| Situation | Recommended response |
|---|---|
| LLM timeout | Use the base ranking after current eligibility validation |
| Unknown resource ID | Reject the invalid output and fall back |
| Invalid response schema | Reject it rather than guessing its meaning |
| Unsupported recommendation claim | Use a controlled explanation template or omit the reason |
| Resource becomes unavailable | Remove it or retrieve another eligible candidate |
| Required review is missing | Hold the resource; do not substitute model judgment |
A syntactically valid response can still contain a false explanation. For strict claims, controlled reason templates based on verified fields are easier to enforce than unrestricted generated text.
Member descriptions and resource text are also data, not instructions. They should not be allowed to expand the candidate set or alter the eligibility rules.
Only send the fields needed for the model’s task. Names, phone numbers, full assessment histories, and unrelated sensitive details generally do not help rank a bounded set of courses. Access controls, retention, and logging practices should reflect the actual sensitivity of the data.
11. Build a Feedback Loop That Measures More Than Clicks
A recommendation request should be traceable to its inputs and decision versions. Useful records include:
recommendationRequestId
resourceId
candidateSource
rankPosition
featureVersion
clusterVersion
ruleVersion
rankingVersion
modelAndPromptVersion
displayedAt
outcomeType
outcomeTimeThese are suggested fields for an auditable implementation, not a claim that every field existed in the first version.
Record actual exposure separately from candidate generation. A resource returned by an API is not necessarily seen by the member.
Track clicks, favorites, bookings, attendance, purchases, refunds, explicit disinterest, and relevant satisfaction feedback. Do not automatically treat a missing click as a negative preference.
Evaluate the whole service
Several measurements complement each other:
- Recommendation coverage and diversity.
- Booking and actual attendance rates.
- Purchase and renewal outcomes.
- Refunds and explicit negative feedback.
- Eligibility violations.
- Empty-result and fallback rates.
- Response latency and model cost.
A long-term outcome such as renewal also has many influences beyond a recommendation. Do not attribute a change to the system without a suitable comparison.
For offline evaluation, respect time: build profiles using information available before the recommendation point and evaluate against later outcomes. Randomly mixing events can leak future purchases into a supposedly historical profile.
Compare against simple baselines such as eligible popular resources and rules plus content matching. If an online experiment is appropriate, keep hard constraints identical across variants and define the primary metric and observation window in advance.
No numerical uplift is claimed here. The purpose of the feedback design is to make future improvements measurable.
12. What the Project Changed About Our View of Recommendation
The initial question was whether K-means could tell us what to recommend. The more useful question became which component should make each decision.
Rules describe explicit business states. Clustering discovers groups whose characteristics still need interpretation. Cohort behavior and other retrieval methods supply candidates. Content representations help match descriptions. LLMs help interpret text and communicate supported reasons.
Those responsibilities work together, but they are not interchangeable.
A member’s peers may like a resource that the member cannot currently use. A semantically similar course may still require a missing review. A well-written explanation cannot make an ineligible recommendation valid.
The most important lesson was to keep preference signals, eligibility, and reviewed suitability separate, then connect them through a traceable recommendation process. That gives the system room to improve without handing its business boundaries to a language model.
Related reading: Reliable RAG with document quality and evidence | Spring AI Alibaba tools, memory, and RAG