[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"me":3,"catalog:en:ml-engineer\u002Fdl-initialization-optimizers":4,"config":169},null,{"field_key":5,"field_name":6,"seniority":7,"topic_key":8,"topic_name":9,"spec_key":7,"spec_name":7,"locale":10,"cell_total":11,"field_total":12,"seniorities":13,"topics":17,"specs":76,"samples":84},"ml-engineer","ML Engineer","","dl-initialization-optimizers","Dl Initialization Optimizers","en",75,1500,[14,15,16],"junior","mid","senior",[18,21,24,27,30,33,36,39,40,43,46,49,52,55,58,61,64,67,70,73],{"key":19,"name":20,"count":11},"cml-distance-clustering-dimreduction","Cml Distance Clustering Dimreduction",{"key":22,"name":23,"count":11},"cml-gradient-boosting-mechanics","Cml Gradient Boosting Mechanics",{"key":25,"name":26,"count":11},"cml-linear-logistic-internals","Cml Linear Logistic Internals",{"key":28,"name":29,"count":11},"cml-probabilistic-models-calibration","Cml Probabilistic Models Calibration",{"key":31,"name":32,"count":11},"cml-svm-kernels-margins","Cml Svm Kernels Margins",{"key":34,"name":35,"count":11},"cml-trees-randomforest-internals","Cml Trees Randomforest Internals",{"key":37,"name":38,"count":11},"dl-architecture-layers","Dl Architecture Layers",{"key":8,"name":9,"count":11},{"key":41,"name":42,"count":11},"dl-losses-output-layers","Dl Losses Output Layers",{"key":44,"name":45,"count":11},"dl-normalization-regularization","Dl Normalization Regularization",{"key":47,"name":48,"count":11},"dl-sequence-attention","Dl Sequence Attention",{"key":50,"name":51,"count":11},"dl-training-dynamics-backprop","Dl Training Dynamics Backprop",{"key":53,"name":54,"count":11},"ml-data-preparation","Ml Data Preparation",{"key":56,"name":57,"count":11},"ml-deployment-serving","Ml Deployment Serving",{"key":59,"name":60,"count":11},"ml-experimentation-reproducibility","Ml Experimentation Reproducibility",{"key":62,"name":63,"count":11},"ml-model-selection-tuning","Ml Model Selection Tuning",{"key":65,"name":66,"count":11},"ml-model-training-evaluation","Ml Model Training Evaluation",{"key":68,"name":69,"count":11},"ml-monitoring-drift","Ml Monitoring Drift",{"key":71,"name":72,"count":11},"ml-problem-framing","Ml Problem Framing",{"key":74,"name":75,"count":11},"ml-scaling-performance","Ml Scaling Performance",[77,81],{"key":78,"name":79,"count":80},"classical-ml","Classical ML",450,{"key":82,"name":83,"count":80},"deep-learning","Deep Learning",[85,103,117,130,143,156],{"id":86,"topic":9,"difficulty":87,"body":88,"options":89,"correct_key":94,"explanation":102},"01a0499c-f0ff-7ede-97ef-f3f17210401c",1,"A fully connected hidden layer is initialised so that every unit in the layer receives exactly the same weight vector and the same bias. Training then proceeds with plain gradient descent. What happens to those hidden units as training continues?",[90,93,96,99],{"key":91,"text":92},"a","Each unit slowly drifts towards a different weight vector, because the mini-batches seen during training differ from one step to the next.",{"key":94,"text":95},"b","They keep receiving identical gradients, so the layer behaves like a single unit copied many times.",{"key":97,"text":98},"c","Only the first unit keeps learning; the rest stay frozen at their initial values.",{"key":100,"text":101},"d","The units diverge as soon as the non-linearity is applied, because ReLU responds differently to each unit's pre-activation.","Units that share the same incoming weights compute the same pre-activation for every input, so the loss is an identical function of each of them and back-propagation hands them identical partial derivatives. Identical parameters plus identical gradients means identical updates forever, so the layer's whole output space collapses onto one direction and the extra width buys nothing at all.",{"id":104,"topic":9,"difficulty":105,"body":106,"options":107,"correct_key":100,"explanation":116},"01a0499c-f102-780a-b2ea-afa6326bccac",2,"In PyTorch 2.8 on CPU, a two-layer MLP `Linear(3,4) -> ReLU -> Linear(4,1)` has every weight matrix and every bias set to exactly zero. One batch is pushed through, an MSE loss is computed and `backward()` is called. Which parameters end up with a non-zero gradient?",[108,110,112,114],{"key":91,"text":109},"Both weight matrices, because the input batch itself is non-zero and reaches the first layer unchanged.",{"key":94,"text":111},"The second layer's weight and bias, since the loss is computed directly from that layer's output.",{"key":97,"text":113},"Every bias in the network, while the two weight matrices are the only tensors stuck at a zero gradient.",{"key":100,"text":115},"Only the output layer's bias, which is the single parameter still exposed to the error.","The hidden activations are relu(0) = 0, and the gradient of the second weight matrix is built from those activations, so it vanishes. The gradient that would reach the first layer has to travel back through the second weight matrix, which is also zero, so it vanishes as well. The output bias is the only parameter whose gradient is the raw error itself, which is why an all-zero network never leaves that fixed point.",{"id":118,"topic":9,"difficulty":87,"body":119,"options":120,"correct_key":91,"explanation":129},"01a0499c-f104-70aa-a86e-4a7484065712","Xavier (Glorot) initialisation draws the weights of a layer from a distribution whose variance is tied to the layer's shape. Which quantity does that variance target?",[121,123,125,127],{"key":91,"text":122},"Roughly 2 divided by the sum of fan_in and fan_out, so that the forward and backward requirements are traded off against each other.",{"key":94,"text":124},"Roughly 2 divided by fan_in alone, matching the number of incoming connections that are summed into each unit's pre-activation value.",{"key":97,"text":126},"Roughly the reciprocal of the batch size, so the variance shrinks as batches get bigger.",{"key":100,"text":128},"Roughly 1 divided by the product of fan_in and fan_out, which keeps the total weight energy fixed.","The Glorot derivation asks for the variance of the activations to be preserved going forward, which wants 1\u002Ffan_in, and for the variance of the gradients to be preserved going backward, which wants 1\u002Ffan_out. A layer cannot satisfy both unless it is square, so the rule takes the harmonic compromise 2\u002F(fan_in + fan_out). Batch size never enters the derivation because it is a property of the data pipeline, not of the layer.",{"id":131,"topic":9,"difficulty":105,"body":132,"options":133,"correct_key":97,"explanation":142},"01a0499c-f104-7beb-9a48-ef2cb287a3b8","He (Kaiming) initialisation uses a variance of about 2\u002Ffan_in for a layer followed by ReLU, whereas the same derivation without an activation would ask for 1\u002Ffan_in. Where does the extra factor of 2 come from?",[134,136,138,140],{"key":91,"text":135},"From the fact that ReLU has a slope of 2 on the positive side, which doubles the signal that passes through it.",{"key":94,"text":137},"From the two weight matrices involved in one back-propagation step, one for the forward pass and one for the backward pass.",{"key":97,"text":139},"ReLU zeroes out roughly half of a symmetric pre-activation distribution, halving the mean square of the layer output, so the weights are scaled up to compensate.",{"key":100,"text":141},"It compensates for the bias vector, which contributes a second independent source of variance to the layer output.","For a zero-mean pre-activation z, the mean square of relu(z) is exactly half the mean square of z, because the negative half of the distribution is mapped to zero and the positive half is passed through untouched. Without correction the signal's second moment would halve at every layer and decay geometrically with depth. Doubling the weight variance restores it, which is exactly what the 2\u002Ffan_in rule does.",{"id":144,"topic":9,"difficulty":105,"body":145,"options":146,"correct_key":91,"explanation":155},"01a0499c-f105-76f0-b297-066b733c1f62","A 12-layer ReLU network with 256 units per layer is initialised with Xavier instead of He, and the mean square of the activations is measured layer by layer on the very first forward pass. What pattern does that measurement show?",[147,149,151,153],{"key":91,"text":148},"The mean square shrinks roughly by half at every layer, so by the deepest layer it is a tiny fraction of the input value.",{"key":94,"text":150},"The mean square grows steadily with depth, roughly doubling each layer, because Xavier is too aggressive for rectified units.",{"key":97,"text":152},"The mean square is unchanged from layer to layer, because Xavier and He differ only in the backward pass.",{"key":100,"text":154},"The mean square stays flat for the first few layers and then grows without bound once ReLU starts saturating in the deeper part of the stack.","For a square layer Xavier gives a weight variance of 1\u002Ffan_in, which is exactly half of what a rectified layer needs, and ReLU then throws away half of the second moment on top of that. The two effects compose into a factor of about one half per layer, which is a geometric decay in depth. A measured run over twelve layers ends several orders of magnitude below the input, which is why deep rectified stacks want the larger He scale.",{"id":157,"topic":9,"difficulty":87,"body":158,"options":159,"correct_key":100,"explanation":168},"01a0499c-f107-79e2-81fc-91bd48244f09","In plain stochastic gradient descent the update applied to a parameter is minus the learning rate times its gradient. If the learning rate is doubled while everything else is held fixed, what happens to that single update?",[160,162,164,166],{"key":91,"text":161},"Its length doubles and its sign flips, since the update is a subtraction.",{"key":94,"text":163},"Its length is unchanged but it rotates towards the steepest coordinate of the gradient.",{"key":97,"text":165},"Its length quadruples, because the loss is a quadratic function of the step size.",{"key":100,"text":167},"Its length doubles and its direction is unchanged.","Plain SGD scales the negative gradient by a single scalar, so the learning rate controls only how far the step goes, never where it points. Doubling a positive scalar doubles the magnitude and leaves the direction alone. The quadratic behaviour of the loss shows up in how much the loss changes, not in how large the parameter update is.",{"fields":170,"seniorities":392,"interview_shapes":393,"locales":398,"oauth":400,"question_count":403,"coach_enabled":404,"jd_match_enabled":404},[171,196,216,233,257,270,289,308,330,349,364,386],{"key":172,"name_tr":173,"name_en":173,"sort":87,"specializations":174},"backend","Backend",[175,178,181,184,187,190,193],{"key":176,"name":177,"field":172},"general","Genel",{"key":179,"name":180,"field":172},"go","Go",{"key":182,"name":183,"field":172},"python","Python",{"key":185,"name":186,"field":172},"java","Java",{"key":188,"name":189,"field":172},"csharp","C#\u002F.NET",{"key":191,"name":192,"field":172},"nodejs","Node.js",{"key":194,"name":195,"field":172},"php","PHP",{"key":197,"name_tr":198,"name_en":198,"sort":105,"specializations":199},"frontend","Frontend",[200,201,204,207,210,213],{"key":176,"name":177,"field":197},{"key":202,"name":203,"field":197},"javascript","JavaScript",{"key":205,"name":206,"field":197},"typescript","TypeScript",{"key":208,"name":209,"field":197},"react","React",{"key":211,"name":212,"field":197},"vue","Vue",{"key":214,"name":215,"field":197},"angular","Angular",{"key":217,"name_tr":218,"name_en":218,"sort":219,"specializations":220},"fullstack","Fullstack",3,[221,222,223,224,225,226,227,228,229,230,231,232],{"key":176,"name":177,"field":217},{"key":179,"name":180,"field":172},{"key":182,"name":183,"field":172},{"key":185,"name":186,"field":172},{"key":188,"name":189,"field":172},{"key":191,"name":192,"field":172},{"key":194,"name":195,"field":172},{"key":202,"name":203,"field":197},{"key":205,"name":206,"field":197},{"key":208,"name":209,"field":197},{"key":211,"name":212,"field":197},{"key":214,"name":215,"field":197},{"key":234,"name_tr":235,"name_en":235,"sort":236,"specializations":237},"devops-cloud","DevOps \u002F Cloud",4,[238,239,242,245,248,251,254],{"key":176,"name":177,"field":234},{"key":240,"name":241,"field":234},"aws","AWS",{"key":243,"name":244,"field":234},"gcp","GCP",{"key":246,"name":247,"field":234},"azure","Azure",{"key":249,"name":250,"field":234},"kubernetes","Kubernetes",{"key":252,"name":253,"field":234},"terraform","Terraform",{"key":255,"name":256,"field":234},"linux","Linux",{"key":258,"name_tr":259,"name_en":259,"sort":260,"specializations":261},"ai-engineer","AI Engineer",5,[262,263,264,267],{"key":176,"name":177,"field":258},{"key":182,"name":183,"field":258},{"key":265,"name":266,"field":258},"llm-rag","LLM\u002FRAG",{"key":268,"name":269,"field":258},"mlops","MLOps",{"key":271,"name_tr":272,"name_en":273,"sort":274,"specializations":275},"database","Veritabanı","Database",6,[276,277,280,283,286],{"key":176,"name":177,"field":271},{"key":278,"name":279,"field":271},"postgresql","PostgreSQL",{"key":281,"name":282,"field":271},"mysql","MySQL",{"key":284,"name":285,"field":271},"mongodb","MongoDB",{"key":287,"name":288,"field":271},"redis","Redis",{"key":290,"name_tr":291,"name_en":292,"sort":293,"specializations":294},"mobile","Mobil","Mobile",7,[295,296,299,302,305],{"key":176,"name":177,"field":290},{"key":297,"name":298,"field":290},"ios-swift","iOS (Swift)",{"key":300,"name":301,"field":290},"android-kotlin","Android (Kotlin)",{"key":303,"name":304,"field":290},"flutter","Flutter",{"key":306,"name":307,"field":290},"react-native","React Native",{"key":309,"name_tr":310,"name_en":311,"sort":312,"specializations":313},"security","Güvenlik","Security",8,[314,315,318,321,324,327],{"key":176,"name":177,"field":309},{"key":316,"name":317,"field":309},"appsec","AppSec",{"key":319,"name":320,"field":309},"offensive-pentest","Offensive \u002F Pentest",{"key":322,"name":323,"field":309},"cloud-security","Cloud Security",{"key":325,"name":326,"field":309},"devsecops","DevSecOps",{"key":328,"name":329,"field":309},"blue-team-incident","Blue Team \u002F Incident",{"key":331,"name_tr":332,"name_en":333,"sort":334,"specializations":335},"qa-test-automation","QA \u002F Test Otomasyonu","QA \u002F Test Automation",9,[336,337,340,343,346],{"key":176,"name":177,"field":331},{"key":338,"name":339,"field":331},"test-automation","Test Automation",{"key":341,"name":342,"field":331},"sdet","SDET",{"key":344,"name":345,"field":331},"performance-testing","Performance Testing",{"key":347,"name":348,"field":331},"mobile-qa","Mobile QA",{"key":350,"name_tr":351,"name_en":351,"sort":352,"specializations":353},"data-engineer","Data Engineer",10,[354,355,358,361],{"key":176,"name":177,"field":350},{"key":356,"name":357,"field":350},"pipelines-etl","Pipelines \u002F ETL",{"key":359,"name":360,"field":350},"streaming","Streaming",{"key":362,"name":363,"field":350},"warehousing","Warehousing",{"key":365,"name_tr":366,"name_en":367,"sort":368,"specializations":369},"game-dev","Oyun Geliştirme","Game Development",11,[370,371,374,377,380,383],{"key":176,"name":177,"field":365},{"key":372,"name":373,"field":365},"unity-csharp","Unity (C#)",{"key":375,"name":376,"field":365},"unreal-cpp","Unreal (C++)",{"key":378,"name":379,"field":365},"gameplay","Gameplay",{"key":381,"name":382,"field":365},"graphics-rendering","Graphics \u002F Rendering",{"key":384,"name":385,"field":365},"multiplayer-netcode","Multiplayer \u002F Netcode",{"key":5,"name_tr":6,"name_en":6,"sort":387,"specializations":388},12,[389,390,391],{"key":176,"name":177,"field":5},{"key":78,"name":79,"field":5},{"key":82,"name":83,"field":5},[14,15,16],{"junior":394,"mid":396,"senior":397},{"questions":395,"median_sec":3},20,{"questions":395,"median_sec":3},{"questions":395,"median_sec":3},[399,10],"tr",[401,402],"google","github",28050,true]