Conversation
…y-stop, grad clip, binary IO
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Description
ModelBinHeaderand implementedSaveBinaryFile/LoadBinaryFileplus new DLL exportsDN_SaveBinaryStateandDN_LoadBinaryStatethat write/read a compact binary representation (magic + version + topology + weights).X_val,T_val),LoadValidationBatchexport and a generalizedComputeDatasetMSEpath; added tracking ofbest_val_lossand a patience-based early stopping flow that snapshots/restores best weights when validation is used.kScaleInplace, helperApplyGlobalGradNormClippingand integration before optimizer updates; exposedDN_SetGlobalGradNormClip/DN_GetGlobalGradNormClip.TrainSync_Lockedand wiringDN_Trainto synchronous training while preserving the existingStartTrainingAsync_Lockedused byDN_TrainAsync; added train/eval mode and RNG seed control with exportsDN_SetTrainModeandDN_SetSeed.DN_LoadValidationBatch,DN_SetEarlyStoppingPatience,DN_GetBestValidationLoss; also neutralized misleading telemetry by returning NaN for unsupported per-layer diagnostics.ComputeDatasetMSE(shared full/validation evaluation), progress updates now optionally track validation metric for best-MSE reporting, and maintenance of early-stopping counters.GRULayerandRNNLayerare still thin derivations ofLSTMLayer(placeholders) and therefore do not yet provide a distinct GRU or vanilla RNN algorithm in this change.Testing
rg/nl(symbol checks succeeded); commit created successfully (git commitsucceeded).git diff/git status(changes recorded as one modified filekernel.cu).nvcc --version) but the environment lacksnvccso no build/test of CUDA kernels could be performed (no binary compiled or runtime tests executed).Codex Task